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Abstract — This paper presents an algebraic theory of linear 
signal processing. At the core of algebraic signal processing is the 
concept of a linear signal model defined as a triple (.4, At, 
where familiar concepts like the filter space and the signal space 
are cast as an algebra A and a module At, respectively, and 
$ generalizes the concept of the z -transform to bijective linear 
mappings from a vector space of, e.g., signal samples, into the 
module Al. A signal model provides the structure for a particular 
linear signal processing application, such as infinite and finite 
discrete time, or inflnite or finite discrete space, or the various 
forms of multidimensional linear signal processing. As soon as 
a signal model is chosen, basic ingredients follow, including the 
associated notions of filtering, spectrum, and Fourier transform. 

The shift operator q, which is at the heart of ergodic theory 
and dynamical systems, is a key concept in the algebraic theory: 
it is the generator of the algebra of filters A.. Once the shift 
is chosen, a well-deflned methodology leads to the associated 
signal model. Different shifts correspond to infinite and finite time 
models with associated infinite and finite z -transforms, and to 
infinite and finite space models with associated infinite and finite 
C-transforms (that we introduce). In particular, we show that the 
16 discrete cosine and sine transforms are Fourier transforms for 
the finite space models. Other definitions of the shift naturally 
lead to new signal models and to new transforms as associated 
Fourier transforms in one and higher dimensions, separable and 
non-separable. 

We explain in algebraic terms shift-invariance (the algebra of 
filters A is commutative), the role of boundary conditions and 
signal extensions, the connections between linear transforms and 
linear finite Gauss-Markov fields, and several other concepts and 
connections. Finally, the algebraic theory is a means to discover, 
concisely derive, explain, and classify fast transform algorithms, 
which is the subject of a future paper. 

Index Terms — Signal model, filter, Fourier transform, bound- 
ary condition, signal extension, shift, shift-invariant, z-transform, 
spectrum, algebra, module, representation theory, irreducible, 
convolution, orthogonal, Chebyshev polynomials, discrete cosine 
and sine transform, discrete Fourier transform, polynomial 
transform, trigonometric transform, DFT, DCT, DST, Gauss- 
Markov random field, Karhunen-Loeve transform 
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I. Introduction 

The paper presents an algebraic theory of signal processing 
that provides a new interpretation to linear signal processing, 
extending the existing theory in several directions. Linear 
signal processing is built around signals, filters, z-transform, 
spectrum, Fourier transform, as well as several other funda- 
mental concepts; it is a well-developed theory for continuous 
and discrete time. In linear signal processing, signals are 
modeled as elements of vector spaces over some basefield, 
e.g, the real or complex field, and filters operate as linear 
mappings on the vector spaces of signals. 

The assumption of linearity has made the theory of vector 
spaces, or linear algebra, the predominant mathematical disci- 
pline in linear signal processing. This paper proposes that the 
basic structure in linear signal processing actually goes beyond 
vector spaces and linear algebra. The algebraic theory that we 
describe will show that this structure is better exploited by the 
representation theory of algebras, which is a well established 
branch of algebra, the theory of groups, rings, and fields.^ 
We will show that by appropriate choices for the space of 
signals, the space of filters, and the filtering operation — what 
we call the signal model — the algebraic theory captures within 
the same general framework many important instantiations of 
linear signal processing, namely: linear signal processing for 
infinite and finite discrete time; Uneiir signjil processing for 
infinite or finite discrete "space;" and linear signal process- 
ing for higher order linear models, e.g., separable and non- 
separable linear signal processing on infinite and finite lattices 
in two or more dimensions. To get a better understanding and 
appreciation for what we mean, we expand on some of these 
examples in the next subsection. 

Remark. This paper focuses on discrete parameter (time 
or space) finite or infinite linear signal processing, which 
for the sake of brevity will simply be referred to as signal 
processing and abbreviated by SP. Much of the paper extends 
to continuous parameter SP, but this will not be considered 
here. 

A. Overview 

The basic idea: Signal models. Consider Table I, ignoring 
for the time being the bold-faced entries and focusing first on 
the second and third columns labeled infinite time and finite 
time. Rows 2 to 5 represent the four basic concepts in SP: 
the 2;-transform, filters, signals, and the Fourier transform. 
Column 2 recalls that, for infinite discrete time SP, we have the 
well defined concept of 2;-transform and that signals and filters 
(in their 2;-transform representation) are described by power 
series in the variable z~^. Filtering becomes multiplication of 
series, and the associated Fourier transform is the well known 
DTFT (discrete time Fourier transform). 

When only a finite number N of samples is available, we are 
in the domain of discrete finite time SP, which is considered 
in the third column in the table. The Fourier transform is 
the well known DFTjv (discrete Fourier transform). However, 
attempting to extend the infinite time case, column 2, to 

'The word algebra describes the discipUne as well as an algebraic structure 
(namely a vector space that is also a ring, to be defined later). 
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TABLE I 

1-D DISCRETE INHNITE AND FINITE TIME AND SPACE SIGNAL PROCESSING AS FOUR INSTANTIATIONS OF THE GENERAL ALGEBRAIC THEORY. THE 

BOLDED CONCEPTS ARE SUPPLIED BY THE ALGEBRAIC THEORY. 



generic theory 


infinite time 


finite time 


infinite space 


finite space 


$ = "^-transform" 


^-transform 


finite z-transfonn(s) 


C-transfonn(s) 


finite C-transform(s) 


A = algebra of filters 


series in z~" 


polynomials in z~" 


series in T„ 


polynomials in Tn 


M = ^-module of signals 


series in 


polynomials in 


series in C„ 


polynomials in Cn 


T = Fourier transform 


DTFT 


DFTs 


DSFTs 


DCTs/DSTs 



the finite time case, column 3, by simply truncating the z- 
transform to obtain polynomials as signals and filters leads 
to problems. Namely, if signals and filters are polynomials 
S{z^^) and H{z~^) of degree iV — 1, then their product is in 
general of higher degree. In other words, the space of signals 
(polynomials of degree A'^ — 1) is not closed under this notion 
of filtering. The solution, which is well-known (e.g., [1]), casts 
filtering, as multiplication modulo z~^ — 1, 



H{z-^)S{z-^) mod (z-^- 1) 



(1) 



Correspondingly, signals and filters are now in the space of 
"polynomials in z~^ modulo {z~'^ — 1)," which is denoted 
by C[z~^]/(z^^ — 1) and is called a polynomial algebra. The 
definition of a finite 2;-transform, not found in the hterature, 
is now straightforward; it is bold-faced in Table I and is 
provided by the algebraic theory. Filtering as described in (1) 
is equivalent to the well known circular convolution. 

Besides the DFT, there are numerous other transforms 
available for finite signals, for example, the discrete cosine 
and sine transforms (DCTs and DSTs) considered in the fifth 
column in Table I. They have been successfully used in image 
processing, so, intuitively, we refer to them as associated to 
"space" signals, in contradistinction to time signals. Note that 
"space" here is one-dimensional (1-D), not necessarily 2-D, 
since the DCTs/DSTs are 1-D transforms. We will discuss in 
more detail below what we mean by space. More importantly, 
we go back to Table I and ask for the DCTs and DSTs: 
What are their analogues of the finite z-transform, signals, and 
filters? Likewise, since the DCTs/DSTs are finite transforms, 
we consider the corresponding infinite "space" analogues in 
column 4, again asking what are the appropriate notions of 
^-transform, signals, filters, and Fourier transform for infinite 
space signals. 

The algebraic theory presented in the paper identifies the 
basic structure to answer these questions as hinted at in Table 1 
and provides the bold-faced entries, which will be defined in 
subsequent sections. In other words, the algebraic approach 
leads to a single theory that instantiates itself to the four right 
columns in Table I and, furthermore, to several other columns 
corresponding to existing or new ways of doing time and space 
SP. 

Central in the algebraic theory of SP is the concept of the 
signal model. It is defined as a triple {A,M., $) (see the first 
column in Table 1): 

• ^ is the chosen algebra of filters, i.e., a vector space 
where multipUcation (of filters) is also defined. 



algebra 



vector 
space 



bijective 



mapping 



filters 



operates on 



module 



signals 



Fig. 1. The central concept in the algebraic theory of signal processing is 
the signal model, which is a triple of an algebra, an associated module, and 
a bijective hnear mapping (all bolded) from a vector space of signal samples 
into the module. Defining this mapping fixes the notion of filtering. 



signals, filters 
spectrum 

frequency reponse 
Fourier transform 



signal model 




Fig. 2. Defining the signal model fixes well-defined notions of signals, filters, 
spectrum, frequency response, Fourier transform, and the other ingredients 
for SP. 



• Al is an A-module of signals, i.e., a vector space whose 
elements can be multiplied by elements of A. We say 
that A operates on M through filtering. 

• <I> generalizes the z-transform. It is defined as a bijective 
mapping from a vector space V of signal samples into 
the module M of signals (see Figure 1). 

The vector space V is a product space of countable or finite 
many copies of, say, the real numbers M or the complex 
numbers C, so that elements of V are series or vectors of 
signal samples. The purpose of the bijective mapping $ is to 
assign a choice of filtering, which is given by the operation 
(multiplication) of A on ^A. We will show that once we fix a 
signal model, apphcation of the well-developed representation 
theory of algebras provides a systematic methodology to derive 
the main ingredients in SP, such as the notions of spectrum, 
Fourier transform, and frequency response (see Figure 2). 

To be specific, we consider briefly two illustrations of the 
signal model for finite SP. 

For finite time, the signal model is given by (we set a; = z~^ 
for simpUcity) A = M = C[x]/{x^ - 1) and 

$ : s ^ Snx" e M, 

0<n<N 

for s = (soj • • • , Siv-i) gV = C"' (assuming complex valued 
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definition of shift(s) - 



signal model 



Fig. 3. A signal model can be derived from the definition of the shift 
operation in two steps that we call linear extension and realization. Using this 
procedure, we can derive signal models that match our intuition by properly 
defining the shift. Many shifts besides the standard time shift are possible. 



signals). This $ is the finite ^;-transform indicated in Table I. 
As we saw in this table, the Fourier transform associated with 
this signal model is the DFT. 

We now consider a second signal model, a finite space 
model, given by a different polynomial algebra: A = M. = 
C[x\/Tn{x), and 

$ : s ^ SnTn{x) e M, 

0<n<N 

where T„ are Chebyshev polynomials of the first kind. We will 
show that the corresponding Fourier transform for this model 
is the DCT, type 3. 

Derivation of signal models: Shift and boundary condi- 
tions. These two examples illustrate how we can fill Table I to 
obtain a consistent set of SP concepts based on the concept of 
a signal model. We address now the question of which algebras 
and modules are implicitly assumed in common instantiations 
of SP and why. This question is important because it will lead 
to a method to develop signal models beyond the ones shown 
in Table I. 

A first high-level answer to this question is provided by 
the algebraic theory: Every signal model {A,M,^) that has 
the shift invariance property has necessarily a conmiutative 
A. If the model is in addition finite, then A has to be a 
polynomial algebra. As examples we saw above the models 
for the DFT and DCT, type 3. To obtain a more detailed 
answer to the question which signal models occur in SP, we 
explain how to derive signal models from basic principles, 
niimely from a chosen definition of the shift operator. The 
shift plays a fundamental role in many areas including ergodic 
theory, random processes and statistics, dynamical systems, 
and information theory. As an abstract concept, once there is 
a group structure, e.g., [2], shifts can be defined for time or 
space, or in multiple dimensions. In the algebraic theory, the 
shift has a particularly simple interpretation: it is the generator 
of the filter algebra. Specifically, we describe a procedure that, 
starting from the definition of the shift, produces infinite and 
finite signal models, and that reveals the degrees of freedom 
that are available in this construction (see Figure 3). This 
procedure provides two important insights: 

1) How to derive signal models based on shifts other than 
the standard time shift; and 

2) The role of boundary conditions and signal extensions in 
finite signal models. 

For example, regarding 1), when starting with an abstract 
definition of the standard 1-D time shift {q is the shift operator, 
❖ the shift operation, tn are discrete time points) 

time: qotn = tn+i, (2) 

we obtain the well-known time models with the associated 
infinite and finite 2;-transform. The very same procedure, when 



• s-» 

(a) 1-D time shift 



in—1 in 

(b) 1-D space shift 



...0 



Cn 

• S-» 



in—1 in in+1 
(c) 1-D generic next neighbor shift 

Fig. 4. Examples of 1-D shifts considered by the algebraic theory. 



applied to a different definition of the shift, namely to what we 
call the 1-D space shift. 



space: qotn = \{tn-i +tn+i), 



(3) 



leads to the infinite and finite C-transform (that we wiU define) 
and, in the finite case, to the DCTs and DSTs (see Table I). 
In other words, different shifts lead to different signal models 
with different associated Fourier transforms; in particular, the 
DCTs or DSTs are Fourier transforms in this sense. 

Other shifts are possible (see Figure 4), and our method- 
ology produces the corresponding signal model and thus the 
appropriate notion of filtering (or convolution), spectrum, and 
Fourier transform for each of them. The method is the same for 
higher dimensional signal models. For example, in 2-D, two 
shifts have to be considered; possible choices are shown in 
Figure 5. The first two lead to the known separable 2-D time 
and 2-D space models, whereas the remaining two choices 
produce novel 2-D signal models (and thus associated notions 
of z-transform, filtering, and Fourier transform) for the finite 
spatial hexagonal and quincunx lattice respectively, [3], [4]. It 
turns out that the signal models arise again from polynomial 
algebras (and thus they are shift invariant), but in two variables 
in this case. 

Regarding 2), the role of boundary conditions and signal 
extension, our signal model derivation explains why they 
are unavoidable (under certain assumptions) and what the 
choices are. For example, why is the periodic extension the 
usual choice for finite time, and why is the symmetric or 
antisymmetric extension an appropriate choice for the DCTs 
and DSTs? This insight is very relevant when deriving the 
novel 2-D models mentioned above. Also, it is interesting to 
note that for example in finite time other signal extensions 
besides periodic are possible, which then produce different 
signal models and thus a different associated "DFT." 

Fast algorithms. One important application of the algebraic 
theory is in the discovery, derivation, and classification of 
fast transform algorithms. There are many different transforms 
used in signal processing (e.g., DFT, DCTs/DSTs, discrete 
Hartley transforms, and variants thereof) and because of their 
importance there are hundreds of publications on their fast 
algorithms. Most of these algorithms are derived by ingenious 
manipulation of the transform coefficients. These derivations, 
however, are usually tedious and provide no insight into the 
structure nor the existence of these algorithms. Further, it is not 
clear whether important algorithms may not have been found. 
The exception is the DFT, for which the theory of algorithms is 



(a) Two 2-D time shifts (separable) (b) Two 2-D space shifts (separable) (c) Two 2-D hexagonal space shifts (d) Two 2-D quincunx space shifts 

(non-separable) (non-separable) 



Fig. 5. In 2-D signal models are derived fi-om 2 shifts. Examples are shown here; the operation of the two shifts in each case is represented by soUd and 
dotted arrows, respectively (scaling factors are omitted). The first two choices of shifts lead to separable models, the others do not. 



well-understood due to early works like [5], [6], [7], [8] and 
others. As a result, very accessible standard books on DFT 
algorithms are now available for application developers [1], 
[9], [10], [11]. We will show that by extending ideas from the 
work on DFTs, the theory of algorithms becomes a natural 
part of the algebraic theory of signal processing. 

The basic idea is to derive algorithms from the signal model 
underlying a transform rather than from the transform itself. 
We briefly sketch how it works in a simple case. We consider a 
signal model with A = A4 = C[x]/p{x). To derive algorithms 
for the associated Fourier transform we first state what T 
actually does in this case. Namely, !F decomposes the signal 
module into its irreducible components, called its spectrum. 
This is akin to decomposing vector spaces into invariant 
subspaces with respect to some linear mapping. In the case 
of a polynomial algebra this decomposition is an instantiation 
of the Chinese remainder theorem ( CRT) and looks as follows: 

^ : C[x]/p{x) ^ 

C[x]/{x - ao) e . . . e C[x]/{x - ttN-i). (4) 

Here dcg(p) = N and the q:„ are the zeros of p, assumed to be 
distinct. The important point is that each of the summands on 
the right side has dimension 1, i.e., C[x]/p{x) is fully decom- 
posed. Intuitively, algorithms are now derived by performing 
this decomposition in steps. This is possible for example, if 
p{x) = q{r{x)) decomposes (note that this is different from 
factorization). In this case, we can perform (4) in two steps 
using again the CRT, namely 

C[x]/p{x) C[x]/{r{x)-Po)(B...®C[x]/ir{x)-(3K- 
C[x]/{x - ao) ® . . .Q)C[x]/{x - aN-i) 

Here, the Pk are the zeros of q{x) and deg{q) = K. A general 
theorem that we already showed in [12] provides an algorithm 
for T in this case. 

For the DFT, p{x) = x^ - 1 = {x^)^ - 1 indeed 
decomposes if N = KM and the method yields the famous 
Cooley-Tukey FFT For the DCT, type 3, p{x) = Tn{x) = 
Tk{Tm{x)) and we obtain algorithms of a structure very 
similar to the Cooley-Tukey FFT [13]. Interestingly, most of 
them are novel. Using the same method we can derive many 
known and novel algorithms for all 16 DCTs/DSTs [14] and 
for other transforms in forthcoming papers. 

The idea of decomposition generalizes to higher dimensions. 
For example, reference [15] derives a Cooley-Tukey type 
algorithm for the new discrete triangle transform for finite 
spatial hexagonal lattices introduced in [3]. This transform is 



a Fourier transform for a finite 2-D signal model based on the 
definition of shifts in Figure 5(c). 

Using the algebraic approach we wiU also generalize the 
well-known prime-factor FFT and Rader FFT, by identifying 
the algebraic principles they are based on. We already showed 
first ideas along these lines in [12]. 

Algebraic theory: Forward and inverse problems. In this 
paper, we apply the algebraic theory to address, in a sense, a 
forward problem and an inverse problem: (1) Forward prob- 
lem: Derive a signal model {A,M.,'^) for a given application 
from basic principles, and find the appropriate SP concepts 
like filtering, convolution, spectrum, signal extension, Fourier 
transform and its fast algorithms, and others; (2) Inverse 
problem: Given a linear transform (say the DCT, type 3), find 
the corresponding signal model, for which the transform is a 
Fourier transform. From the signal model, we can then derive 
all SP concepts mentioned in the forward problem above, 
including fast algorithms for that transform. 

The algebraic theory enables the solution of both problems. 
In the paper we focus on linear, shift invariant signal pro- 
cessing since this is a simple context where the algebraic 
approach can provide immediate meaningful results. Also, 
rather than assuming more general constructions, e.g., discrete 
spaces, metric spaces, or Polish spaces, we restrict ourselves 
to scalar signals that are rational, real, or complex valued — 
this more restrictive approach still applies to many relevant 
linear transforms and signal models as we consider here. 

Accessibility of the algebraic theory. Algebra is not among 
.the mathematical disciplines commonly taught or used in 
'SP. However, the major parts of the algebraic theory of SP 
can be developed from working knowledge with series and 
polynomials and from basic linear algebra techniques, each of 
which is common knowledge in SP. We will introduce several 
algebraic concepts; however, they describe existing concepts in 
SP. For example, referring to the space of filters and signals 
as an algebra and a module, respectively, does not impose 
new structure; rather, it makes explicit the structure commonly 
adopted. 

Summary: Scope of the algebraic theory. In summary, 
the scope of the algebraic theory of SP as we present it and 
plan to further develop it can be visualized as an expansion 
of Table I. 

First, we develop the general theory (first column) and 
then we apply or instantiate this theory to expand the table 
with additional columns by deriving relevant signal models. 
We start with 1-D SP and fiU in the models for most of 
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the existing spectral transforms^, which include practically 
aU known trigonometric transforms. Future papers wiU then 
further expand the table through separable and non-separable 
SP in higher dimensions. 

Second, we expand Table I along the rows, for the generic 
theory and for all signal models introduced. We start again 
with the most basic concepts such as spectrum, frequency 
response, Fourier transform, diagonalization properties, and 
convolution theorems. Forthcoming papers will then develop 
the algebraic theory of fast transform algorithms, subsam- 
pling, uncertainty relations, filterbanks, multiresolution analy- 
sis, frame theory, and other important concepts in SP. 

B. Background: Algebra in Signal Processing 

In this section, we review prior and related work using alge- 
braic techniques in signal processing, and discuss the particular 
thread of research that led to the work in this paper. Existing 
work is mostly focused on the derivation of algorithms for the 
DFT (or a few other transforms) or on the construction of very 
specialized signal processing schemes, such as Fourier analysis 
on groups. This restriction to specialized applications has put 
algebra and representation theory essentially in the "blind 
spot" of signal processing theory. The situation is different 
for the related field of system theory, which we discuss first. 

Algebraic system theory. This paper describes an algebraic 
view to explain the mathematical structure underlying signal 
processing. Kalman in his seminal work on linear system 
theory (see chapter 10 in his co-authored book [16J from 
1968) went beyond vector spaces. His concern was to show 
that "the entire theory of the regulator problem . . . depends 
on the algebraic properties of the [system] matrices F, G, and 
H satisfying these two conditions." The conditions Kalman 
refers to are the controllability and observability conditions 



rank [G, re, • • • .F^'-^G 
rank , ,■ ■ ■ , {F^f H^' 



= n 
= n, 



(5) 
(6) 



where n is the dimension of the state space 
{F : n X n, G : n X m, H : I X n). He interpreted (5) 
"algebraically by viewing it as a condition for generating a 
module over polynomials in the matrix F." 

Kalman used the algebraic approach to study the realization 
problem in Unear systems. This is an inverse problem: How 
to go from a suitable rational function that is the external 
system description to the state variable model, the matrices 
F, G, and H above, that is the internal system description. 
Important concepts, like invariance and invariant subspaces, 
more appropriately dealt with in the framework of algebras and 
modules, play an important role in reaUzation theory and are 
common staple of linear systems and control theory, starting 
with the early work of Basile and Marro [17] and Wonham 
and Morse [18] (see also the critical paper by Willems and 
Mitter [19]). The unpublished work in the PhD thesis of 
Johnston [20] and Fuhrman [21] and his subsequent work, for 
example [22, ch. 10], as well as others, have further developed 
the algebraic theory of linear systems with emphasis on 

^The impatient reader may want to check Table XV for the result. 



the system realization problem. Extensive work by Fuhrman 
focuses again on linear time invariant systems because their 
algebraic properties can provide significant insight. Many 
more references and authors have explored these ideas in linear 
systems and related areas; it is not our intention nor can we 
even provide here a fair coverage of this literature, so we 
will not discuss this topic further and refer the readers to the 
relevant literature. 

In parallel with Kalman's and Fuhrman's module perspec- 
tive on linear systems, this paper explores the theory of alge- 
bras and modules, i.e., the representation theory of algebras, 
as a basic framework for digital linear signal processing. In 
realization theory and in the algebraic linear system theory the 
shift operator, its matrix representations, and its irreducible 
components play a central role. Likewise, in the direct and 
inverse SP problems that we are interested in (see above), 
the shift operator and the decomposition of modules in irre- 
ducibles play a very important role in identifying the spectrum, 
the linear transform, and its fast algorithms associated with a 
linear signal model. 

Algebraic methods for DFT algorithms. The advent of 
digital signal processing is often attributed to the rediscovery 
of the fast Fourier transform (EFT) by Cooley and Tukey in 
1965 [23], [24]. In the following years, the recognition of 
the importance of fast algorithms for the DFT led to the 
first — and to date arguably most important — appUcation of 
algebraic methods in mainstream signal processing. Namely, 
it was already known in the 19th century that the DFT can 
be described in the framework of the representation theory 
of groups and, more specifically, can be related to the cyclic 
group. This connection was used to derive and explain existing 
EFT algorithms including the Cooley-Tukey EFT [25], [26], 
[5], [8], [7], but is also the foundation of Winograd's seminal 
work on the multiplicative complexity of bilinear forms in 
general and the DFT in particular. This work provided an 
entirely new class of DFT algorithms that, surprisingly and 
among other things, showed that the DFT can be computed 
with only a linear number of (non-rational) multiplications [6], 
[27], [28], [29]. 

Fourier analysis and fast Fourier transforms on groups. 
The connection between the DFT and the cyclic group made 
it natural to explore the appUcability of other "group Fourier 
transforms" in signal processing. The general topic of Fourier 
analysis on groups dates back to the early days (19th century) 
of the representation theory of groups with major contributions 
by Gauss, Frobenius, Burnside, Brauer, and Schur. For the in- 
finite (additive) cyclic group of integers Z, the area of Fourier 
analysis is equivalent to the theory of Fourier series, which is 
standard in functional analysis [30], [31] and in discrete-time 
signal processing. Generahzations to other infinite, commu- 
tative groups have also been extensively studied (e.g., [32]). 
In signal processing, general commutative finite groups were 
considered already in [25] (including fast algorithms). The first 
proposition of non-commutative groups is due to Karpovsky 
[33]. This development raised the question of fast algorithms 
for these transforms, starting a new area with the pioneering 
work of Beth [8], [34]. The field was further extended by 
Clausen [35], [36], [37], and by the large body of work by 
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Rockmore et al., which shaped the field as it stands today; 
examples include [38], [39], [40], [41]. 

The infinite cychc group Z and the finite cyclic group 
lead to discrete-time signal processing and finite time signal 
processing with periodic boundary conditions, respectively. 
Every finite commutative group is a direct product of cyclic 
groups, and can thus be viewed as a multi-dimensional torus, 
which leads to separable multi-dimensional finite signal pro- 
cessing. Beyond that, Fourier analysis on non-commutative 
finite groups has found next to no applications in signal 
processing. There are a few notable exceptions. The work 
by Diaconis [42], [43] identifies the symmetric group as the 
proper structure to analyze ranked statistical data. DriscoU 
and Healy develop Fourier analysis for signals given on 
the 2-sphere (the surface of a three-dimensional ball) [44]. 
More recently, Foote et al. propose groups that are wreath 
products for signal processing [45], [46]. These groups offer 
a structure that naturally provides a multi-resolution scheme 
for finite signals. Intriguingly, these wreath product group 
transforms generaUze the well-known Haar transform that is 
different from the one in standard wavelet theory. We want to 
mention that groups do play a role in standard wavelet analysis 
but in a sense different from the work above [47]. Finally, 
and somewhat unrelated, we want to mention the work by 
ShokroUahi et al. [48], which provides a striking application of 
the representation theory of groups in multiple-antenna signal 
processing. 

Background of this paper. The particular thread of research 
that led to the present paper can also be traced back to the 
work of Beth on fast Fourier transforms for groups [8] and 
to the quest of a general theory of fast transform algorithms. 
While group theory provides a set of transforms and (in many 
cases) their fast algorithms, many of the transforms used in 
signal processing, such as the DCTs and DSTs, were not 
captured in this framework. In the search for the algebraic 
properties of these transforms, Minkwitz, in his PhD. work, 
relaxed the idea of signals on groups to signals on sets on 
which groups act via permutations (similar to [45] mentioned 
above) and found that indeed some of the DCTs could be 
described as generalized group Fourier transforms this way. 
Furthermore, he showed that, in these cases, fast algorithms 
for these transforms can also be constructed by pure algebraic 
means [49], [50]. Minkwitz' work was further extended by 
Egner and PUschel in their PhD. work including an automatic 
method to analyze a given transform for group properties, 
and, in the affirmative case, to automatically construct a 
fast algorithm [51], [52], [53]. Application to various signal 
transform showed that several, but not all transforms, could 
be characterized this way [51]. Further, among the many 
existing DCT/DST algorithms, only few could be derived and 
explained this way. The conclusion was: if the DCTs/DSTs 
had a defining algebraic property, it had to be outside the 
group framework. This paper addresses precisely this issue and 
show that the DCTs and DSTs can all be characterized in the 
framework of polynomial algebras instead of group algebras. 
Valuable hints in the search for this structure were provided 
by [54], [55], [56]. Using the polynomial algebras underlying 
the DCTs and DST, we showed how to derive, explain, and 



classify most of the existing fast DCT/DST algorithms [12] 
and we also derived new fast algorithms, not available in the 
literature or found with previous methods [13], [14]. 

C. Organization 

This paper is divided into three main parts: 

• Algebra and signal processing; 

• Discrete infinite and finite signal models and trigonomet- 
ric transforms; and 

• Algebraic signal models, graphs, Markov chains, and 
Gauss-Markov random fields or processes. 

Algebra and signal processing. The first part consists of 
Sections II-IV. In Section II, we introduce background on 
algebras and modules and establish their coimection to signal 
processing. We define the concept of signal model and explain 
the algebraic interpretation of the shift and shift-invariant 
signal models. In the finite case, these models correspond 
to polynomial algebras, for which the signal processing is 
developed in Section 111. Finally, Section IV provides a short 
sunnmary of the first part of the paper and connects to the 
second part. 

Discrete infinite and finite signal models and trigono- 
metric transforms. The second part of the paper consists 
of Sections V-XIV. In each section (except the last two) 
we derive an infinite or finite signal model from a definition 
of the shift, following the same high-level steps. Thus, these 
sections are organized very similarly, with subsections corre- 
sponding to the derivation of the signal model, the derivation 
of spectrum and Fourier transform, the model's visualization, 
diagonalization properties of the Fourier transform, and, op- 
tionally, convolution theorems, orthogonal Fourier transforms, 
and other important properties of the model. Section XIII 
gives an overview of the finite signal models presented so 
far. Section XIV concludes this part with the algebraic theory 
of higher-dimensional signal models. 

Algebraic signal models, graphs, Markov chains, and 
Gauss-Markov random fields or processes. The third part 
of the paper, consisting of Sections XV and XVI, investigates 
the general connection between signal models based on poly- 
nomial algebras, graphs, Markov chains, and Gauss-Markov 
random fields. In particular, we show under which conditions 
a random field is equivalent to a signal model, and thus the 
concepts of Fourier transform and Karhunen-Loeve transform 
coincide. 

Finally, we offer conclusions in Section XVII. 
In the appendix we provide some additional mathematical 
background that is used in this paper. 

II. Algebras, Modules, and Signal Models 

This section introduces the mathematical framework of 
the algebraic theory of signal processing. As said in the 
introduction, by signal processing, or SP, we mean linear 
signal processing. We start with relating algebras and modules 
to signal processing. Then, we introduce modules and algebras 
more rigorously and establish the connection between basic 
concepts in the representation theory of algebra (i.e., the theory 
of algebras and their modules) and SP. Next, we formally 
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define the concept of a signal model, which is at the heart 
of the algebraic theory, as a triple of an algebra, a module, 
and a bijective map. Instantiation of the signal model leads 
to different ways of doing SP (as discussed in the context of 
Table I). After that, we identify in the algebraic theory the 
role of the shift(s) as the generator(s) of the filter algebra 
and explain that shift-invariant signal models are precisely 
those with a commutative filter algebra. Finally, we introduce 
the notion of visualization of a signal model as a graph and 
introduce module manipulation as a useful tool in working 
with signal models. 

This section is mathematical by nature. We recommend that 
the reader consider it as a reference for the more concrete 
examples developed in the sequel. 

A. Motivation 

In SP (linear signal processing), the set of signals is con- 
sidered to be a vector space, like C^, the set of one-sided 
complex valued sequences. With vector spaces, signals can be 
added and can be multiplied by a scalar a (from the base 
field), to yield a new signal. Formally, 

signal -|- signal = signal, 
a ■ signal = signal. 

The structure of a vector space gives access to the notions 
of dimension, basis, linear mapping, and subspace. Because 
of our focus on linear SP, and unless stated otherwise, we 
restrict the discussion to vector spaces that are product spaces 
of the reals numbers R or the complex numbers C, i.e., or 
C^, where the indexing set / is either countable or finite and 
the underlying field is either R or C. 

From a mathematical point of view, the structure of vector 
spaces is simple. Namely, any two vector spaces defined over 
the same field and of the same dimension are isomorphic, i.e., 
structurally identical. As we explain next, the signal models 
used in SP are actually algebraic objects that have more 
structure than vector spaces. Indeed, in SP, signals interact 
with linear systems^, commonly called filters. 

This is represented in block diagram form by 

signal — 



filter 



signal 



(7) 



The operation of filters on signals imposes additional struc- 
ture on the signal space, namely, that of a module. This 
additional structure casts linear signal processing in the frame- 
work of the representation theory of algebras. To recognize 
the additional structure, we first denote the filter operation as 
multiplication • and represent (7) as 

filter • signal = signal. 

The multiplication is meant here in an abstract sense, i.e., 
it can take different forms depending on the representation 
of filters and signals, e.g., convolution (in the time domain) 
or standard multiplication (in the 2;-transform domain) or any 

'We only consider single-input single-output linear (SISO) systems in 
this paper. Extensions to multiple-input multiple-output (MIMO) systems are 
under research. 



Other adequate form, as long as certain properties are satisfied, 
e.g., the distributivity law: 

filter • ( signal -|- signal ) 
= filter • signal -|- filter • signal. 

Furthermore, filters themselves can be combined to form new 
filters: namely added, multiplied, and multiplied by a scalar a 
from the base field, i.e.. 



filter -I- filter = 
filter • filter = 
a ■ filter = 



filter (parallel connection), 
filter (series connection), 
filter (amplification). 



Multiplication of two filters and the multiplication of a filter 
and a signal, though written using the same symbol •, are 
conceptually different. 

Parallel connection and amplification, or, more generally, 
linear combinations of filters makes the filter space (as the 
signal space) a vector space. But, multiplication of filters or 
multiplication of a signal by filters shows that there is more 
structure in the linear SP that goes beyond vector spaces. 

Mathematically, the above structure is described by regard- 
ing the filter space as an algebra A that operates on the 
signal vector space A4, thus making the signal space M. an 
A-module: 



set of filters/linear systems = an algebra A 

set of signals = an ^-module A4 



The signal module as an ^-module allows for "mul- 
tiplication," i.e., filtering, of an element of the module (the 
signal) by an element of the algebra (the filter). Given an 
algebra A and an associated ^-module, the well-developed 
mathematical theory of ^-modules (or representation theory 
of algebras) provides besides filtering access to a larger set of 
concepts than linear algebra. Examples include the notions of 
spectrum, irreducible representation (i.e., frequency response 
as we explain later), and Fourier transform. 

We remark that the structures of the signal space M and 
the filter space A are actually different. For example, signals 
can not be multiplied, while filters can, and filters operate on 
signals, but not vice-versa. 

This paper addresses questions Uke: (1) How to connect 
existing signal processing concepts and theory with algebraic 
concepts and theory? (2) Which algebras and modules natu- 
rally occur in signal processing and why? (3) What benefits 
can we derive from this connection? 

We already considered the first item by revealing the al- 
gebraic nature of filters and of signals. Next section extends 
the connection between algebra and signal processing to in- 
clude concepts like spectrum, frequency response, and Fourier 
transform. In Section II-C we then introduce the definition 
of a signal model, which formalizes the connection between 
algebras, modules, and linear SP. 

To address the second item we will characterize at a very 
high level those algebras that provide shift-invariant filters. 
In the finite case, i.e., for finite-length signals, this will lead 
to polynomial algebras. Later, we derive the algebras and 
modules associated with infinite and finite discrete-time and 
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infinite and finite discrete-space SP. As mentioned in Section I, 
the distinction between time and space is not due to 1-D versus 
2-D but due to directed versus undirected in a sense that will be 
defined rigorously later. In the finite case these constructions 
lead naturally to the discrete Fourier transform (DFT) and 
the discrete cosine and sine transforms (DCTs/DSTs), respec- 
tively. Further, we will reveal the algebrjiic structure behind 
practically all known trigonometric transforms and also extend 
this class by introducing new transforms. 

Regarding the third item, we mention that the algebraic the- 
ory provides, as already mentioned, the common underpinning 
for many different infinite and finite linear signal processing 
schemes, showing, for example, that the spectral transforms 
arise as instantiations of the same common theory. Further, the 
connection between algebras/modules and signal processing 
goes in both directions, namely, in the direct direction (see 
discussion in Section I) specifying an algebra A and an A- 
module A4 provides the ingredients to develop extensions 
to the existing signal processing schemes. As examples, we 
briefly discussed in Section I non-separable 2-D SP on a finite 
quincunx or hexagonal lattice. Finally, also briefly mentioned 
in Section I, is the subject future papers (e.g., [14]), which ex- 
tend the algebraic theory of signal processing to the derivation 
and discovery of fast algorithms. The algebraic theory makes 
the derivation of algorithms concise and transparent, gives 
insight into the algorithms' structure, enables the classification 
of the many existing algorithms, and enables the discovery 
of new algorithms for existing transforms and for new Unear 
transforms. 

B. Algebras, Modules, and Signal Processing 

In this section, we introduce the concepts from the theory 
of algebras and modules that are needed to formulate the 
algebraic theory of SP. Formal definitions are in Appendix I. 
For a more thorough introduction to module theory, we refer 
to, e.g., [57], [58], [59]. We will provide a short dictionary 
between algebraic and signal processing concepts. Section II- 
C formally defines the concept of a signal model. 

Algebras (filter spaces). We denote by C the set of complex 
numbers. A C-algebra ^ is a C-vector space that is also a 
ring, i.e., the multipUcation of elements in the C-vector space 
is defined (see Definition 38 in Appendix I for the formal 
definition). Examples of algebras include C, the set C"^" of 
complex n X n matrices, and the set of polynomials C[a;] in 
the indeterminate x and with coefficients in C. We can choose 
a base field different from C, for example the real numbers 
M or the rational numbers Q, and will do so occasionally but 
then explicitly say so. Note that it has to be a field, otherwise 
the vector space structure is lost. Since ^ is a vector space, 
concepts that only require this structure, such as basis and 
dimension, are well-defined. 

As we mentioned in the previous section, algebras serve as 
spaces of filters in signal processing, with the filters (or hnear 
systems) being the elements of the algebra. Thus, in this paper, 
elements of the algebra are to be considered filters. To ease 
this identification, we represent the elements of the algebra 
by h, a common symbol for filter in signal processing. 



We say that elements hi, . . . ,hk & A generate A, if every 
element in A can be written as a multivariate polynomial or 
series in hi, . . . ,hk, or, equivalently, by repeatedly forming 
sums, products, and scalar multiples from these elements. Most 
algebras considered in this paper are generated by one element; 
its special role will be discussed in Section II-D. 

Modules (signal spaces). If .4 is an algebra, then a (left) 
.4-module M is a vector space, over the same base field (we 
assume C) as A, that admits an operation of A from the left"'. 
We write this operation as multiplication: 

{h,s) ^ h- se M, for h€A, se M. (8) 

This ensures that M is closed or invariant under the operation 
of A. In addition, this operation satisfies several properties 
including, for h, h' ^ A, s, s' £ M, a E C, 

h- {s + s') = h- s + h- s', 

h-{as) = a{h-s), (9) 
h'-{h- s) = {h' ■h)-s. 

The formal definition of an >l-module is given in Definition 39 
in Appendix I. 

The multiplication h ■ s of elements of h E A with elements 
of s & M. captures in the algebraic theory of signal processing 
the concept of filtering: the elements /i e ^ are the filters and 
the elements s e Al are the signals. We emphasize that the 
definition of a module always implies an associated algebra; 
viewed by itself, a module is only a vector space. 

In the algebraic theory of signal processing, modules are the 
signal spaces and elements of modules are the signals. To help 
with this identification, we denote signals with the symbol s 
whenever possible. In this paper, we focus on discrete linear 
signal processing^ and thus on modules in which the elements 
have the form of series or linear combinations s = X^ie/ 
where the index domain / is discrete (e.g., / could be finite 
or J = N, Z). The coordinate vector of a signal s is written 
as s = (si\i e /). The base vectors bi (which, as elements of 
Ai, are signals) are in signal processing called impulses (also 
unit pulses or delta pulses). Given a filter h G A, its impulse 
response, i.e., the response of the filter h to an input which is 
the impulse bi € M, is given by h ■ bi. The definition of M 
assures that h ■ bi is well-defined and again a signal, i.e., an 
element of A4. 

Regular module (filter space = signal space). An impor- 
tant example of a module is the regular >l-module. This is 
the case when the module and the algebra are equal as sets: 
A4 = A, with the multiplication operation in (8) given by the 
ordinary multiplication in A. Even though the sets A4 = A 
may be equal, their algebraic structures are different; for 
example, elements in M cannot be multiplied. In this paper, 
we wiU distinguish between elements in A4 and elements in 
A. 

Representations (filters as matrices). As a consequence 
of the properties in (9), every filter h G A defines a hnear 

*It can also be defined with the algebra operating from the right, which 
leads to a dual theory. 

'We deliberately use the term "discrete" instead of "discrete-time," since 
one of our goals is to identify signal models for discrete space. 
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mapping on A4: 



h ■ s. 



(10) 



filter algebra 



filter algebra 



If A4 has finite dimension n and we choose a basis b = 
{bo, ■ ■ ■ , &ri-i)^ in M, this linear mapping is represented by a 
complex n X n matrix Mh, which is a matrix representation 
of the filter h. As usual with linear mappings, is obtained 
by applying h to each base vector bi, the coordinate vector of 
the result hbi is the ith column of Mh- 

By constructing Mh for every filter /i e A we obtain a 
mapping </> from the algebra A (the set of filters) to the algebra 
of n X n matrices C"^": 



(j): C"^", h ^ (pih) = Mh. 



(11) 



The mapping is a homomorphism of algebras, i.e., a mapping 
that preserves the algebra structure (see Definition 40 in 
Appendix I). In particular, 

4){h + h')^<t>{h) + 4>{h') and <p{hh') ^ (P{h)(f){h'). 

<f) is called the (matrix) representation of A afforded by the 
A-module A4 with basis b. The representation is fixed by the 
choice of the module A4 and the basis b of M.. Different 
choices of M and b lead to different matrix representations 
of A. However, 4> is independent of the basis chosen in A. 

The set of matrices 4){A) is an algebra that is structurally 
identical to A. Correspondingly, if s = X]"=o^ ^i^i ■= i-^-^ 
s = (.so: ■ . • , Sn-i)"^ is the coordinate vector for s, then the 
abstract notion of filtering (multipUcation of s e jVt by /i e ^) 
becomes in coordinates the matrix-vector multiplication: 



h - s ^ (f){h) ■ s. 



(12) 



This coordinatization of filtering also shows the fundamental 
difference between signals and filters; namely, in coordinates, 
signals become vectors, and filters (as linear operators on 
signals) become matrices. 

If is not of finite dimension, but still discrete, i.e., 
consisting of infinite series, say, of the form s = J2iez 
still obtain a matrix representation 0, but the matrices are now 
infinite. If is continuous, there is no matrix representation, 
rather, an operator representation. 

Irreducible submodule (spectral component). If M is 
an ^-module, then a sub vector space A4' < A4 is an ^- 
submodule of if M.' is itself an ^-module. Equivalently, 
A4' is closed or invariant under the operation of A. Most 
subvector spaces fail to be ^-submodules, because, intuitively, 
the smaller the vector space A4' is, the harder it is to remain 
invariant under A. 

A submodule Ai' < Ai is irreducible if it contains no 
proper submodules, i.e., no submodules besides the trivial 
submodules M = {0} and M itself. 

In particular, every one-dimensional submodule Ai' is irre- 
ducible and is a simultaneous eigenspace of all filters h £ A, 
i.e., hs = XhS for all s e with a suitable Xh S C. 

In signal processing, an irreducible module corresponds to 
a spectral component. This will become clear in the next 
paragraph. 

*We write bases always as lists in parentheses, not as sets in curly braces, 
since the chosen order of the base vectors is important. 



operates on 



M 



signal space 



spectrum A4ui, co €W 



Fig. 6. A visualization of the concept Fourier transform, which decomposes 
the ^-module M into a direct sum of irreducible (minimal) ^-invariant 
subspaces, i.e., ^-submodules. The latter are called the spectrum of M. 




(So,) 



-+- 



Fig. 7. Visualization of the convolution theorem (21) in the finite case. On 
the left, filtering is in coordinates equivalent to <p{h)s. In the Fourier domain 
on the right, the filter operates on invariant subspaces; thus the matrix is now 
block-diagonal with the blocks 4>oj{h) being the frequency response of h. 



Module decomposition, spectrum, Fourier transform. In 

signal processing, Fourier analysis involves the decomposition 
of signals into spectral components. The algebraic theory gives 
a general definition. Namely, Fourier analysis decomposes an 
^-module into a direct sum^ of irreducible ^-submodules 
M.^^, where cj € W (some index domain). We call the 
corresponding mapping A the Fourier transform for the A- 
module M: 



A: M 

s 



(13) 



The existence of such a decomposition is not guaranteed and 
depends on A and A4. 

In (13), each submodule Mu; is called a spectral component 
of the signal space M, and each projection G Ad^^ is 
a spectral component of the signal s. The collection of aU 
Aiuj and all s^, to €W,is called the spectrum of M and s, 
respectively. The spectrum of s is a list with index set W, and 
as such can be viewed equivalently as a function on W: 

See Figure 6 for a visualization of the Fourier transform. 
Its definition is intuitive from a signal processing point of 
view; the Fourier transform decomposes the signal space into 
the smallest components that are invariant, no matter how the 
signal is filtered. Another requirement usually imposed is that 

^By "direct sum" of vector spaces, we usually mean the more general 
"outer" direct sum rather than the "inner" direct sum. See Definition 42 in 
Appendix I for an explanation. 
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A be invertible, i.e., that every signal can be reconstructed 
from its spectrum. 

The Fourier transform is an ^-module homomorphism (see 
Definition 41 in Appendix 1), which means that A{h ■ s) = 
h-A{s) fov h € A, s € M. In words, this means that filtering 
in the signal space A4 is equivalent to parallel filtering in the 
spectrum (as visuaUzed in Figure 6), or 



{h ■ s)a; = h ■ S(^,, for all a; e W. 



(14) 



If A is invertible, then A{h-s) = h-A{s) also yields a general 
convolution theorem: 



h-s = A-\h-A{s)). 



(15) 



The definition of A in (13) (and thus also the convolution 
theorem (15)) is coordinate-free, i.e., formulated indepen- 
dently of the bases chosen in M. and the Ai^^, to € W. We 
coordinatize A in two steps. First, we choose bases on the 
right side, i.e., in the Al^j's. Then A takes the form 

^dim(A1ti,) 

(16) 



A : 



M 

s 



(By abuse of notation we use the same letter A.) Again, the 
spectrum, now in its coordinate form, can be viewed as a 
function on W: 

Choosing also a basis in A^, we obtain the coordinate form 
of the Fourier transform, denoted by J^:^ 



^ : s {Saj)u,ew- 



(17) 



We refer to both A and as Fourier transform, and we also 
refer to {Sui)uiew as the spectrum of s. In signal processing, 
the Fourier transform is usually thought of either as A in (16), 
or as !F in (17). 

In particular, if M is of finite dimension n and if the Fourier 
transform exists, then W is also finite and usually chosen as 
W = {0, . . . , k — 1), k < n. In this case, .F is an n x n matrix. 
If fc = n, then all irreducible modules are of dimension 1. 

Irreducible representations (frequency response). In the 
decomposition (13), each irreducible A4ui affords an irre- 
ducible representation (j)^, of A with respect to a chosen basis 
bi^. Namely, if s^^ e C'^™^-^") is the coordinate vector of the 
spectral component s^j S A4u>, then for every filter hGA,hy 
(12), 



(h ■ s)^ ^ h ■ <^ 4>uj{h) ■ Su 



(18) 



where (/>a;(/i) is a dim(A^^j) x dim(A^^j) matrix. For a fixed 
filter h, the collection 



(19) 



is in signal processing called the frequency response of h 
and can be viewed as a matrix-valued function on W . If 
the irreducible submodule is one-dimensional, then, by 
invariance, it is an eigenspace for every h & A, and ^oj{h) is 
the corresponding eigenvalue. 

^Note that we do not write : C^'™'-^) ^ C'^^^-^"'), since 

for dim(yvi) = oo, the coordinate space for M does not need to have the 
form C^'MM) . e.g., it could be j " 



We could call the mapping 

h {(f)u>{h))u,ew 

that maps every filter to its frequency response the Fourier 
transform of A (w.r.t. the module A4), but we refrain from 
doing so in this paper and reserve the term Fourier transform 
to the decomposition of the module or of signals into their 
spectrum. 

If M is of finite dimension n and with k spectral compo- 
nents A4i of dimension di, < i < k, then ^ = n. If we 
choose bases in A4 and in the Aii's, then takes the form in 
(17) and is an n x n matrix. Filtering in M is in coordinates 
given by the matrix 4>{h). Filtering in the decomposed module 
®o<i<fe coordinates given by 



where (/'^(/i) is a x di matrix and 
A®B = diag(^, B) = 



B 



(20) 



denotes the direct sum of matrices. Since maps the under- 
lying vector spaces, we get 



T ■ (t>{h) ■ = Mh) • 



ffc-i 



(21) 



which is visualized in Figure 7. In other words, the matrices 
J^(j){h)f~^ are block diagonal, with the sizes of the blocks 
given by the dimensions di of the irreducible modules A^i. In 
particular, if all A4i are one-dimensional, di = 1, then k = n 
and T(j){h)J^~^ is diagonal, i.e., (21) gives the diagonahzation 
property of F" and is a coordinatized version and a special case 
of the convolution theorem (15). 

Summary. We summarize the correspondence between al- 
gebraic concepts and signal processing concepts in Table II. 
The signal processing concepts are given in the first column 
and their algebraic counterparts in the second column. If 
we choose bases in the occurring modules, we obtain the 
corresponding coordinate version given in the third column. 
In coordinates, the algebraic objects, operations, and mappings 
become vectors and matrices and thus allow for actual com- 
putation. This is the form used in signal processing. However, 
the coordinate version hides the underlying module structure, 
which cannot be easily recovered if it is not known beforehand. 

Example: infinite discrete time. Often, in linear signal 
processing, the index time is assumed to be continuous and 
taking values on the real Une M or nonnegative reals or 
to be discrete and taking values in N (non-negative integers) 
or Z. We will refer to these cases as the infinite continuous 
or infinite discrete time case. We develop here an example for 
the algebraic framework for infinite discrete time and consider 
filters and signals to be represented in the z-domain. The z- 
transform is denoted by For example, the set of all two- 
sided infinite series, called Laurent series, is represented by 

^(C^) = {E„eZ«n^"1«neC}. 

We find a suitable algebra of filters. A first attempt is to 
choose the vector space $(C^), but this set is not an algebra 
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TABLE II 

Correspondence BETWEEN discrete signal processing concepts and algebraic concepts. 





aL^CULalK^ L/UllUCUL (^L/UVJi villi dlC 11 CC^ 


111 UtJUlUlllcllCa 




filter 


h & A (algebra) 


J / T \ _ .'T^ T ^ 7 

4>ih) e C-"^-* 




signal 


s = X) 5<&i G ^ (^-module) 


s = isi)iei e 




filtering 


h ■ s 


</>(/l) • s 




impulse 


base vector bi G M 


bi = (...,o,i,o,...)reC^ 




impulse response oi h E A. 


h-bi e M 






Fourier transform 








spectrum of signal 


A(s) = {su,)ujew = a; 






frequency response of h e A 


n.a. 


(</'uj(/l))a;eW = ti* l-» </>a>(fe) 





TABLE m 

The generic element for the sets used in Figure 8. From top to 
bottom: Laurent series, power series, rational function, 
Laurent polynomial, polynomial. 



symbol generic element 

C{z~^) p{z~^)/q(z~^), p, g polynomials 

— e<n<k '^"'^ , On S 
C[2-l] Eo<n<fe«n2-", aneC 



since multiplication, i.e., convolution is in general not possible. 
Namely, if a, a' e $(C[Z]), then 

aa' = J2{J2 (^ka'e)z-". (22) 

The inner sum has infinitely many terms and does not converge 
in general. Thus we need to consider a smaller space. 

To do this, we use Figure 8, which shows in a block diagram 
the sets that are commonly considered in infinite discrete 
time signal processing and their algebraic structure. We use 
the following mnemonics. We indicate in each box the set it 
represents by a symbol such as $(C^). Table III explains these 
symbols by giving a generic element. Solid boxes in Figure 8 
are vector spaces, algebras are marked bold, and dashed boxes 
are multiplicative groups that are not vector spaces. In each 
box we indicate a short mathematical description (M) of the 
set it represents or a characterization of the set if viewed as 
a set of filters (F) or if viewed as a set of signals (S) (see 
also the legend in Figure 8). The middle column is for two- 
sided series; the right column is for one-sided series; and the 
left column is for series expansions of rational functions^. 
The arrows between different boxes depict various inclusion 
relationships (the tip of the arrow points towards the smaller 
set). Finally, the dark- gray area indicates for which modules 
the Fourier transform exists. 

We now return to the search for an algebra. We have 
ruled out already above the set <1>(C^) as a possibility for 
A, because multipUcation of two-sided infinite series is not 
possible in general. Among the sets in Figure 8, the next 

'Note that for a rational function, various expansions are possible in general. 
Also note that we use the symbol C{z~^) for rational functions and there 
expansions likewise. 



largest candidate for A is the set of right-sided series A = 
^{C^). In mathematics, this is the set oi formal power series, 
written as C[[2;~^]], and it is an algebra (in (22) the inner sum 
has only finitely many terms and is thus always well-defined). 
In looking for an ^-module A4 for this algebra, it is easy to 
verify again that the set $(C^) is not an ^-module (again, 
for a G A, a' E Ai, the inner sum in (22) will have infinitely 
many terms; thus, the product does not exist in general). The 
next candidate for the corresponding ^-module is the regular 
module A4 = A. However, there are problems with choosing 
for algebra of filters ^ = C[[z~^]]. First, this set contains only 
causal or one-sided filters. Second, the Fourier transform for 
the regular module M = A does not exist (note that it lies 
outside the dark-gray area in Figure 8). 

The next largest choice for A is the space of bounded-input, 
bounded-output (BIBO) stable systems A = $(i?^(Z)). This 
is the set usually chosen in signal processing. As module for 
A, we can consider again the regular module = ^; in 
alternative, we can actually choose a larger space of signals. 
A well-known theorem'" states that ^(£p{Z)) is a $(£^(Z))- 
module for p > 1. Thus, we could attempt to select as a 
module the largest of such sets A4 = $(^°°(Z)). The problem 
is that, again, as it is well-known, the Fourier transform does 
not exist. The choice commonly made in signal processing 
is Al = $(^^(Z)), the space of finite-energy signals". With 
these choices, we emphasize that for the discrete time example 
under consideration we chose the sets A and Ai to be different, 
i.e., the sets of filters and of signals have not only different 
algebraic structures (one is an algebra and the other is a mod- 
ule) but, as sets, they are actually different, a fact that is rarely 
explicitly stated in the signal processing literature. Finally, we 
note that because of the need for efficient implementations 
of filters, usually only the smaller algebra C{z~^)n^{£^ {Z)) 
of BIBO stable series that are also expansions of rational 
functions is considered. 

In summary, with these choices A = $(^^(Z)) and M = 
^{i^{Z)), filters h and signals s take the form h = H{z) = 
J2nez^nZ~~"' and s = S{z) = Z]„gz ■Sn-^"", respectively. 
The basis elements z~"' are called delays if they are filters 
and impulses if they are signals. 

^"Theorem 45, provided with proof in Appendix n. 

"We could actually choose larger modules M = ^{ff{Z)), p < oo, if 
the proper definition of convergence for the Fourier transform is chosen [30], 
[31]. However, we will work here with finite energy signals. 
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F ; filter/system description 
S : signal description 



Fig. 8. A block diagram identifying the algebraic structure of sets commonly used in discrete-time signal processing. Note that the module property is not 
captured in this diagram since it depends on the choice of algebra. 
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The representation (j) of A afforded by A4 with basis'^ b = 
(. . . , z~^, z^,. . .) maps filters /i to a doubly infinite matrix 
with Toeplitz structure. 

The next task is to identify the irreducible modules Adui, 
i.e., the spectrum of Ai. It is well-known that each 

E^{z) = J2 6^""^"", toeW= (-7r,7r], (23) 

riGZ 

is a simultaneous eigenvector for all filters h = H{z) e A, 
namely 

H{z)E^{z) = HienE^z), w€W. (24) 

This implies that the one-dimensional space Ai^^ spanned by 
E^^ is an yl-module and irreducible (since of dimension I). 
Further, (24) shows that 

(j}^ : H{z) ^ H{eJ'^) € C 

is the irreducible representation afforded by Mu; if the list of 
length 1 (E^) is chosen as basis. Note that (puih) = H{e^'^) 
is a scalar because A4uj is one-dimensional. 

The corresponding Fourier transform is called the discrete- 
time Fourier transform (DTFT) and, since the Ea,{z) are 
orthogonal, it takes the form 

s = S{z) ^ {S{e^'^)E^{z)}^ew. 

This matches (13), but there is one problem. The spectral 
components E^ are not in M, but only in $(^°°(Z)). So, 
the A4uj are still irreducible ^-modules, but not submodules 
of M. This is one of the problems that can arise in the 
infinite case; in this paper we are mostly concerned with finite- 
dimensional modules where this problem does not occur. In the 
present example however, A still exists, and its coordinatized 
form (16) is the one actually called DTFT in signal processing: 

s = S{z) ^ (5(e^-))^6vy = w ^ 5(e^-), 

where W = (— tt, tt]. A(s) is usually viewed as function 
on the circle.'^ The operation of A on A4, i.e., convolu- 
tion, H{z)S{z) becomes a set of pointwise multiplications 
H{e^'^)S{e^'^) in the Fourier domain. Further, A can be 
inverted, i.e., the signal can be reconstructed from its spectrum. 

Finally, the frequency response for a filter h = H{z) € A 
is given by the collection of all irreducible representations 
evaluated at h: 

{(bUH{z)))^ew = (H{enhew H{eJ^). 

Note that the frequency response of the filter h is obtained 
in the same way as the spectrum of the signal s, namely by 
evaluating at e^'^, i.e., by "applying the Fourier transform" 
to the filter h £ A (we use double quotes since we defined 
the Fourier transform only for A4). This is due to the special 

'^Note that we do not use the term basis in the strictest mathematical sense, 
which requires the linear combination to be finite. However, the notion of basis 
can be generalized to the way it is used here, if the space is a Banach space 
[60], which it is for i'^ or coefficient sequences. 

'^In fact, it is an -function on the circle [32], but this fact is not of 
importance in our discussion. 



Structure of the algebra and module and may be misleading; in 
general, the spectrum (in coordinate form) consists of vectors 
of length (the dimension of M.^;), and the frequency 
response consists of d,^ x d^, matrices (the representations 
afforded by Mu;)- They coincide in dimensionahty only for 
d^ = 1. 

Figure 8 suggests that many other combinations of filter 
algebra and signal module are possible, and this is indeed 
the case. For example, we can keep the signal module M. = 
$(^^(Z)) and restrict the filter algebra to a smaller algebra, 
e.g., to causal FIR filters A = C[z^^]. Choosing now this 
A, we can reduce the signal module, for example, to the 
signals with finite support, M = C[z,z~^], or, as another 
example, to the right-sided signals = ^. In the algebraic 
framework, these would be different signal models; however, 
the associated spectrum and Fourier transform in these cases 
is essentially equivalent to the more general model considered 
above. We will later consider models, which have a sub- 
stantially different notion of spectrum and thus also Fourier 
transform (as example see also Table I). 

At this point we hope to have conveyed to the reader that 
important concepts from discrete-time signal processing are 
equivalent to the more general concepts from the theory of 
algebras and modules. This correspondence enables us to port 
linear signal processing to other algebras and modules. A 
more immediate question, however, is whether other modules 
and algebras are actually used in standard signal processing 
without being explicitly stated. This is indeed the case as 
hinted at in Section I, and the main motivation for developing 
this algebraic theory. Before we consider these models, we 
introduce the central concept in the algebraic theory of signal 
processing: the formal, algebraic definition of a signal model. 

C. Algebraic Definition of Signal Model 

In the previous section we asserted that the assumptions 
underlying SP naturally make the filter space an algebra A 
and the signal space an associated yl-module Ai. Conversely, 
if any yl-module A4 is given, filtering is automatically defined, 
and the well-estabhshed module theory can be applied to 
rigorously derive the spectrum, the Fourier transform, and the 
other concepts in signal processing. 

However, signal processing does not commonly consider 
modules. In particular, signals are not viewed as elements 
of a module, but, in the discrete case considered here, as 
sequences of numbers from the base field over some index 
range. If the index range is fixed, e.g., / = {0, . . . , n — 1}, 
then the corresponding set of signals, e.g., C", naturally is a 
vector space. The question is: How do we formally associate 
a module to this vector space? The answer is given by the 
following definition of a (hnear) signal model, which is the 
central concept in the algebraic theory of signal processing. 

We consider discrete complex signals s, i.e., sequences s e 
of complex numbers over some index range / C Z. The set 
of signals is a vector space V < C^. For finite I = {0, . . . ,n— 
1}, typically, V = c{°'--"-i> = C". If 7 = N, Z, we usually 
consider V = £^{1), V = £'^{1), or V = C^. 
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Definition 1 (Linear Signal Model) Let F < be a vector 
space of complex signals over a discrete index domain I. A 
discrete linear signal model, or just signal model, for y is a 
triple A^,$), where A is an algebra of filters, is an 
^-module of signals with dim(A^) = dim(V"), and 

^■.V^M (25) 

is a bijective linear mapping. If A, A4 are clear from the 
context, we simply refer to <& as the signal model. 

Further, we transfer properties from M to the signal model. 

For example, we say the signal model is regular or finite, if 
A4 is regular or finite (-dimensional), respectively. 

Note that the definition of the signal model has hnearity 
built in (due to the operation of ^ on A^) in accordance with 
the algebraic theory being a theory of linear signal processing. 

Remarks on signal model. Intuitively, a signal model 
endows the vector space V < with the structure of the 
module M as graphically displayed in Figure 1. Via the 
signal model we can then identify s & V with the element 
in the module s = $(s). As a consequence, filtering is now 
well-defined and we get immediate access to all module- 
theoretic concepts introduced in Section II-B, including spec- 
trum, Fourier transform, and several others not yet introduced. 

For example, if Ai is of dimension n with basis b = 
{bo,...,b„-iy'^ and s e C", then 

n-l 

$(s) = s = Y^ Sibi (26) 

defines a signal model for F = C". Conversely, if $ is any 
signal model for V with canonical basis Cj (ith element in Cj 
is 1; all other elements are 0), then the list of all bi = ^(ej) is 
a basis of M. (since $ is bijective) and thus $ has the form in 
(26). In other words, the definition of signal model impUcitly 
chooses a basis in A4 and $ is dependent on this basis. In 
fact, we will later see examples of signal models (associated 
to the DCTs) that differ only in this choice of basis or i.e., 
have the same algebra and module. 

Definition 1 makes it possible to apply different signal 
models to the same vector of numbers. For example, we will 
later learn that by applying a DPT or a DCT to a vector of 
length n one is imphcitly adopting different signal models for 
the same finite-length vector. 

From a strictly mathematical point of view, and in the 
algebraic definition of signal model, the bijection in other 
words, the usual ^-transform in signal processing, serves 
simply to track the basis chosen for the signal module A4. 
This basis determines the operation of the algebra on the vector 
space V. 

We remark that Definition 1 of the signal model and the 
algebraic theory extends to the case of continuous signals. 
However, in this ,we will not pursue this extension and limit 
ourselves to discrete signals. 

As an example, we show next that the z-transform, is 
the linear mapping $ of a signal model in the sense of 

^'^In this paper b will always denote a basis and bi always basis elements, 
which should not be confused with scalars such as Sj, /ij. 



Definition 1. For this reason, we will refer to the linear 
mapping in other signal models as transforms, such as the 
C-transform or the P-transform that we will introduce. 

Example: z-transform. We present the signal model for 
the ^-transform. We choose as algebra 

A = {J2 hnz-'^li. . . , h-uho, /ii, . . . ) e 

the set of aU Laurent series with £^ coefficient sequences, and 
as module 

M = {Y^ . . , S-l, SO, Si, . . . ) G 

the set of all Laurent series with £^ coefficient sequences. A4 
is indeed an ^-module as we discussed in Section II-B. We 
complete the definition of the signal model by identifying the 
bijective linear mapping $ in Definition L The ordinary z- 
transform will do 

s ^ s = ^(z) = X;„ez 

In summary, {A,A4, $) is a signal model for the vector space 
V = £'^{Z). This signal model is, of course well-known and 
the one commonly adopted in mainstream discrete-time signal 
processing. 

After the z-transform is chosen, it becomes clear how to do 
filtering. Namely, if h = H{z) e A and s = S{z) € M, then 
the result r = B.{z) of filtering s with h is simply the product 
of Laurent series 

R{z) = H{z)S{z). (28) 

If we work with the respective coefficient sequences, then the 
nth coefficient r„ of r follows from (28) 

rn = '^hiSn-i- (29) 

iGZ 

The signal model, the operation of ^ on A^, and the choice of 
transform makes clear the definition of (29) or (28). Without 
making explicit the algebraic structure , the origin of (29) as 
filtering is obscured. The problem is that in (29) filtering is 
defined in terms of coordinates with respect to a basis, but the 
basis, which explains the structure of (29), is not provided. 

Signal processing books emphasize the usefulness of the 
2;-transform, since common signal processing operations are 
conveniently expressed in the z-domain. In algebraic terms 
this means that it is more convenient to work with the 
expUcit algebra and module rather than with the vector spaces 
of coefficient sequences. For this reason, we believe it is 
necessary to identify the signal models {A, M. , for all 
the spectral'^ Unear transforms ^, thus identifying as the 
Fourier transform (in the algebraic sense) for A4. This is one 
of the goals achieved by the algebraic theory. 

'^We use the word "spectral" here for transforms such as DFT, DCT, and 
others, to distinguish from other transforms (such as the z-transform), which 
do not compute a spectrum of some sort. 
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D. Shifts, Shift-Invariance, and Commutative Algebras 

So far, the only examples of algebras and modules used 
in signal processing that we provided are those shown in 
Figure 8. These are associated with infinite discrete- time signal 
processing. An important question is which other algebras and 
modules actually occur in discrete signal processing and why. 
It is possible to give a preliminary answer to this question 
by introducing and requiring the concept of shift-invariance. 
We start by understanding what "shift" and "shift-invariance" 
means in our algebraic theory by focusing first on the case 
where only one shift is available, i.e., on 1-D signals'^. Then 
we extend the discussion to multiple shifts. 

Shift. Defining transforms and processes on groups is 
common in many areas. For example, in ergodic theory or 
in dynamical systems, a probability space is associated with 
a mapping, which can be a shift, that can take many different 
abstract forms depending on the underlying space. To be more 
specific, and following [2], the usual model in ergodic theory is 
a probability space (O, !F, V) and a measurable transformation 
T (often called the shift). The set O is assumed to consist of 
infinite or finite duration sequences or waveforms and often 
assumed to be a product space of the real line (or more 
generally a Polish space). Measurable maps / are then defined 
on it. Of particular interests are maps taking into the real 
line or some subset thereof. With such a mapping /, then 
y{t) = f{T^u!) for t in some group gives a sequence or 
waveform for every w e f2. The mapping / produces a shift- 
invariant mapping from sequence (or waveform) to sequence 
(or waveform), which leads to a general theory for general 
alphabets based on when T is measure preserving (and hence 
the processes stationary). This general setup works for time 
shifts and space shifts and most signals likely to be of interest 
in applications. 

This paper considers specific instantiations of this general 
theory and looks for very particular forms of the shift as 
they have been used in linear signal processing, or that may 
explain existing linear transforms or may lead to new linear 
transforms. To achieve this, we show that, in the algebraic 
theory, the shift has a particularly simple interpretation. The 
shift operator is a special filter, and thus is an element'^ x G A. 
Further, it is common to require that every filter h E A he 
expressed as a polynomial or series in the shift operator x. 
Mathematically, this means that the shift operator generates'^ 
the algebra A. Since a similar statement holds also for multiple 
shifts (discussed below): 

shift(s) = chosen generator(s) of A. 



'*In the sequel, we use 1-D and m-D to refer to one-dimensional and m- 
dimensional signals with respect to the number of indexing parameters of the 
signal. For example, a standard time signal is 1-D, while signal like an image 
is a 2-D. We reserve the word "dimension" to refer to the dimension of the 
signal space when viewed as vector space. 

'^We write x instead of to emphasize the abstract nature of the 
discussion. Later, this will enable us to introduce without additional effort 
other shifts as well. 

'*This is not entirely correct, as, in a strict sense, one element x can only 
generate polynomials, not infinite series. However, by completing the space 
with respect to some norm the notion of generating can be expanded. We 
gloss over this detail to focus on the algebraic nature of the discussion. 



Shift-invariant algebras. A key concept in signal process- 
ing is shift-invariance. In the algebraic theory this property 
takes a very simple form. Namely, if a; is the shift operator 
and h a filter, then h is shift-invariant, if, for all signals s, 
h{xs) = x{hs), which is equivalent to hx = xh. Requiring 
shift-invariance for all filters h thus means 

x-h = h-x, for all /i e yl. (30) 

Since x generates A, A is necessarily commutative, and (30) 
is of course guaranteed. Conversely, if .4 is a commutative 
algebra and x generates A, then all filters h E A ctre shift- 
invariant.^^ This observation is simple but crucial, and it also 
holds for multiple shifts (discussed below): 

shift-invariant signal model 4^ A. is commutative 

In particular, shift-invariance is a property of the algebra, 
and not of the chosen module (signal space) in a signal 
model. However, different choices of modules will, in general, 
produce different signal models as we will see later. 

Which algebras are shift-invariant? We can now ask 
which algebras lead to shift-invariant signal models, or equiv- 
alently, which algebras A are commutative and generated by 
one element x? In fact, if A is generated by one element it is 
necessarily commutative; in other words, signal models with 
just one shift are always shift-invariant. This is different in the 
case of multiple shifts discussed below. 

In the case of one shift, we have to identify those alge- 
bras that are generated by one element x. In the infinite- 
dimensional case, we get algebras of series in x or polynomials 
of arbitrary degree in x. In the finite-dimensional case, these 
algebras are precisely the polynomial algebras 

A = C[x]/p{x), p a polynomial of degree n. 

C[x]/p{x) is the set of all polynomials of degree less than 
n with addition and multiphcation modulo p{x). As a vector 
space, A has dimension n. 

Thus, using only shift-invariance as a requirement, we have 
identified one of the key players in the algebraic theory of 
signal processing, namely polynomial algebras. They provide 
the signal models for many transforms, such as the DFT, DCT, 
and others, and for several new transforms. This observation 
motivates our Section III, which develops the general theory 
of signal processing using polynomial algebras by specializing 
the general algebraic theory in Section II-B. 

In the remaining discussion on shift-invariance, we consider 
the situation where several shifts are available and the relation- 
ship between polynomial algebras and group algebras. The 
reader may want to skip this part at first reading and proceed 
with Section II-F. 

Multiple shifts. In general, if m-D signals are considered, 
m shift operators xi,. . . , Xm are available. These may operate 
along different dimensions of the signal as in the usual 
separable case, but can also take different forms as shown 

''The requirement of "x generating A" is indeed necessary as there are 
linear shift-invariant systems that cannot be expressed as convolutions, i.e., 
as series in x; see [61]. 



17 



in Figure 5 for non-separable models that we derived using 
the present algebraic theory. 

The above discussion on one shift is readily extended to 
multiple shifts but there are some differences. Again, the 
xi,. . . , Xm generate A and shift-invariance becomes 

Xi ■ h = h ■ Xi for a\l h G A,l < i < k, 

which is equivalent to A being commutative. We can reduce 
this condition to 

(31) 



X 1 ' Xi • 



In words, a signal model with m shifts is shift-invariant if and 
only if the shifts commute in pairs. 

Commutative algebras generated by m elements include 
multivariate series (e.g., Laurent series in more than one 
variables). 

For an exact classification, we restrict ourselves to alge- 
bras generated by xi,. . . , Xm in the strict sense, i.e., those 
containing only multivariate polynomials, no series. In signal 
processing terms, this is equivalent to A containing only FIR 
filters. In particular, every signal model for a finite set of 
samples, i.e., with dim{M) < oo falls into that class. 

Commutative algebras generated by 3: = (xi, . . . ,Xm) are 
precisely all multivariate polynomial algebras (the notation is 
explained in Appendix I together with the Chinese remainder 
theorem) 

A^C[x]/{piix),...,pk{x)), (32) 

where Pi{x), 1 < i < fc, are polynomials in m variables. In 
words, A is the algebra of all polynomials in m variables with 
addition and multiplication defined modulo the k polynomials 
Pi. Equivalently, A is the algebra of all polynomials in m 
variables, with the restriction that the equations pi{x) = 
• • • — Pkix) = have been introduced. Mathematically, 
(pi(x), . . . ,Pk{x)) is the ideal of C\x] generated by the pi, 
and the polynomial algebra is called a quotient algebra. Note 
that if fc = 1, i.e., p = pi, then we write simply C[x]/p{x) 
instead of C[x]/{p{x)) as we did already above. 

As a remark, we observe that the polynomial algebra A 
can be of infinite or finite dimension. For example, for m = 
1, fc = 0, we get C[x], which is of infinite dimension but with 
countable basis. The primary example in this paper, discussed 
above, is the case m = l,fc = 1, i.e., A = €.[x]/p{x), for 
some finite degree polynomial p{x). This algebra is of finite 
dimension. 

Intuitively, if m is given, we need at least k = m poly- 
nomials Pi in (32) to make the dimension finite. However, 
conversely, choosing k = m polynomials does not guarantee 
the polynomial algebra A to be finite-dimensional, unless fc = 
m = 1. Also, it is known that for m > 1 a polynomial algebra 
can have arbitrary large fc, no matter how the polynomials pi 
are chosen [62]. 

The 2-D signal models referred to in Figure 5, namely 
for spatial signals residing on a finite hexagonal or quincunx 
lattice, are indeed shift-invariant (and regular). The associated 
polynomial algebras have fc = m = 2. 

Next, we briefly discuss Fourier analysis on groups to put 
it into the context of the algebraic theory. 



Fourier analysis on groups. Let G be a finite group. Take 
the elements of the group to be a basis for the following vector 
space 

C[G] = {Y,agg\a,eC}. 

geG 

Clearly, C[G] is a vector space, spanned by the group ele- 
ments. It is also clear that we can define in a standard way 
multiplication of elements in C[G] by using the distributive 
law and the multiplication of group elements. Thus, C[G] is 
an algebra. Another point of view is to regard C[G] as the set 
of complex functions g i-^ ag on the group G. The regular 
module A4 = A = C[G] provides a signal model in the sense 
of Definition 1. Namely, if G has n elements, we can set 
A = M= C[G], and 



$ : 



s 



C[G\ 



(33) 



In particular, both signals and filters are elements of the 
group algebra in this case. The study of the signal models 
{A,M.,^) in (33) is the area of Fourier analysis on finite 
groups (briefly discussed in the introduction), which thus 
becomes an instantiation of the algebraic theory of signal 
processing. 

According to the notion of shift introduced above, we also 
have shift operators in a group G, namely the elements of the 
chosen generating set for G. Note that, unless the group is 
cyclic, G and thus CfG] requires at least two generators, i.e., 
shifts. If G is not commutative, then the generators will not 
commute in pairs, i.e., violate (31). Thus the signal model (33) 
is shift-variant. 

An irmnediate question is how polynomial algebras and 
group algebras differ. Since polynomial algebras are always 
commutative, it is clear that for a non-commutative group 
the associated group algebra cannot be a polynomial algebra. 
On the other hand, it is known that every group algebra for 
a commutative group is a polynomial algebra with a very 
specific structure. Namely, a commutative group G is always 
the direct product of cyclic groups G = Z^^ x • • • x Z^^, 
where Z„. is of size rij and is generated by Xi\ thus we get 

C[G] = C[xi, . . . , xJ/(K*-l|l < i < m}).C[G] = C[xu . . . ,x„]/({a 

(34) 

In the case of one variable (one-dimensional signals), G is 
necessarily cycUc, G = Zn, and we have 



C[Z„] = C[x]/(x" - 1). 



(35) 



This algebra is known to be associated to the DFT (of size n) 
as we will discuss later. Comparing (34) and (35) to general 
polynomial algebras in (32), it becomes clear that only very 
few polynomial algebras are also group algebras. 

Historically, the observation that the DFT is associated 
to C[Zn] spawned a significant effort in developing Fourier 
analysis for other, non-commutative groups and in deriving 
their fast algorithms. The observation that non-commutative 
groups produce shift-variant signal models may explain why 
these groups have to date not found many applications in signal 
processing. 
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In this paper, we investigate polynomial algebras, i.e., shift- 
invariant systems, and we will show that algebras other than 
C[a;]/(a;" — 1), and thus not related to groups, are indeed rel- 
evant in signal processing. Before we identify these algebras, 
we will provide a general discussion of signal processing on 
polynomial algebras <C[x\/p{x) in Section ni that specializes 
the algebraic concepts from Section II-B to this specific case. 




Fig. 9. The intersection of the set of group algebras C[G] and the set of 
polynomial algebras is precisely the group algebras for commutative groups 
G. In particular, for one shift (one variable, one group generator) there is only 
the common element C[Z„] = C[x\/{x'^ — 1)> which is associated to the 
DFT„. 



E. Visualization of a Signal Model 

A given signal model can be visualized by a graph (see 
Appendix I, Definition 44), which provides an intuitive under- 
standing of the model. We will use these visualizations later. 

Definition 2 (Visualization of Signal Model) Assume that a 
signal model A4, $) is given and 



$ : s 



where the 6„ form a basis b of A4. Denote the chosen shift 
operators, i.e., generators, of the corresponding algebra A by 
xi,. . . ,Xn- Further, assume that (p is the representation of A 
afforded by A4 with basis b. Then each 4'{xi) is an infinite or 
finite matrix (which we call shift matrix) and can be viewed 
as the adjacency matrix of a weighted graph Qi. Each of these 
graphs has the same vertices corresponding to b. Thus we can 
join these graphs by adding the adjacency matrices of the Gi 
to obtain a graph Q. 

We call the graph Q a visualization of the signal model 
{A,M,-f). 

Intuitively, the graph provides the topology imposed by 
the signal model. This will become clear by looking at the 
various examples shown in this paper. As a first example, we 
use the only signal model we have identified so far, the z- 
transform defined in (27). The chosen basis in M consists 
of the monomials b = (a;")„gz. The one available shift 
operator x operates on 6 as a; • = x""*"^. In coordinates, 
this means that (p{x) is a doubly infinite matrix with ones on 
the lower diagonal and zeros else. The graph that has (j){x) as 
adjacency matrix is shown in Figure 10. The vertices are the 
base elements 6„, the edges show the shift. 



-1 



,.0 



tLi tAj tAj tXj tAj tAj 

Fig. 10. Visualization of the signal model z-transform as a graph. 



F. Module Manipulation and Commutative Diagrams 

A convenient tool when working with modules are A- 
module homomorphisms (Definition 41 in Appendix I). In 
particular, these mappings are Unear mappings, and, in many 
cases considered in this paper simply base changes within one 
module. Since we always work with explicitly chosen bases, 
in the finite-dimensional case these mappings are represented 
by matrices. Formally, if A^i, A^2 are ^-modules with chosen 
bases, and B is the matrix representing the ^-module homo- 
morphism w.r.t. these bases, then we write 



Ml 



Ml 



Above, we already encountered an example for this arrow 
notation: the first row in (13) maps M to its decomposition; 
w.r.t. chosen bases, the mapping corresponds to the matrix 
form of the Fourier transform T (again, we consider only 
the finite-dimensional case here). If two or more mappings 
between modules are connected, then the associated matrices 
obey the following rule (the Bi are the base change matrices): 



Ml 



Ml 



B2 



Ma 



implies 



Ml M3. 



This identity shows that a given mapping (or matrix) from Mi 
to A^3 can be factorized via a module M2- With the proper 
choice of inserted module, this will be a crucial tool for the 
derivation of fast algorithms [14]. In this paper, we use it for 
the commutative diagrams introduced below. 
Another important identity is 



Ml 



M2 implies M2 — > Mi 



provided that B is invertible. 

The second important algebraic tool is the use of com- 
mutative diagrams, which are built from the above module 
mappings. A typical example looks like 



(36) 




The term "commutative" signifies that different paths connect- 
ing two given modules yield the same associated matrix. For 
example, in the above diagram, we can connect Mi to 4 in 
two different ways: via M2 or via M3. The diagram implies 
the identity 

B4B2 = B^Bi. 

III. Signal Processing on Polynomial Algebras 

In Section 11-D we have learned that shift-invariance leads 
naturally to polynomial algebras A in the signal model 
{A,M,^). In particular, in the case of finite 1-D signal 
models, these algebras are necessarily of the form C[x]/p{x). 
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With this motivation, we investigate what it means to do signal 
processing using these algebras. We do this by specializing the 
general theory from Section II-B to A = C[x]/p{x) focusing 
on the regular case A4 — A. 

We start with the definition of polynomial algebras in one 
variable, then we investigate their spectral decomposition, their 
associated Fourier transforms with diagonahzation properties, 
and formulate a general convolution theorem. The results 
provide the general framework for signal processing using the 
DFTs, DCTs, and DSTs. 

A. Polynomial Algebras in One Variable 

Let p{x) be a polynomial of degree deg(p) = n. Then, 
A = C[x]/p{x) = {h{x) I dcg(/i) < n}, the set of residue 
classes modulo p, is an algebra with respect to the addition of 
polynomials, and the polynomial multiplication modulo p. We 
call A a polynomial algebra (in one variable). Polynomial 
algebras are always cyclic, i.e., generated by one suitable 
element, usually chosen to be x, which is then called the shift 
operator (see Section II-D). This means that all elements in A 
are obtained by repeatedly forming powers, sums, and scalar 
multiples from x. In other words, the elements are polynomials 
in X as implicit from the definition above. 

Example. As a simple instructive example, we consider 
p{x) ~ {x—l){x+l) = x^ — 1. Multiplying the two elements 
x,x + 1 G A, we get 



x{x + l)=a;^+a; = a; + l mod (a;^ — 1) 



(37) 



by replacing with 1. We read (37) as "x^ +x is congruent 
(or equal) to a;+ 1 modulo x"^ — 1." Thus, we do not use "mod" 
as an operator, as in 

(x^ + x) mod (x^ — 1) = X + 1, 

but to denote equality of two polynomials modulo a third 
polynomial. 



B. Signal Model 

We choose as vector space V = C", as algebra a polynomial 
algebra A = C[x]/p(x), dcg(p) = n, and as module the 
regular^" module Ai = A. Further, we choose a basis b ~ 
{po, . . . ,Pn-i) of M. As we show next, this provides a 
finite, n-dimensional signal model {A,M, $) in the sense of 
Definition 1. Namely, if s = (sq, • • • e C", we can 

define the bijective linear map $ as 



sePe- 

o<e<n 



(38) 



Note that $ depends on the chosen basis b. The basis elements 
Pi are the unit impulses in Ai, i.e., those with coordinate 
vector se = for £ ^ i, and Si = 1. The impulse response of 
a filter h E A for the impulse pi is hpi S Ai. 

The mapping $ endows the vector space C" with the 
structure of the >l-module A4 with basis b. Thus, we can now 

^"A more general discussion would consider arbitrary modules for A; 
however, it turns out that most of the signal models actually used in signal 
processing have regular modules. The non-regular cases occurring in this paper 
will be studied as they arise. 



identify C" and A4, which becomes our signal space. $ in 
(38) is the equivalent of the z-transform for this model. 

Example. Continuing our previous example, we choose the 
basis 6 = {1, x} in A1 = ^ = C[a;]/(x^ — 1). A signal model 
for the vector space is now provided by 

C'^ ^ M, {so,Si) So + Six. 



C. Filtering 

As seen before, the signal model defines filtering on the 
signal space A4 through the operation of ^ on A^. The algebra 
A is the space of filters and the ^-module A4 the space of 
signals. We mention again that even though the sets M and 
A are equal, their algebraic structure (i.e., the role or structure 
assigned to the sets) is not. For example, A operates on A4, 
not vice-versa, and filters (elements of A) can be cascaded, 
i.e., multiplied, which signals (elements of M.) cannot. 

We can represent filtering in either a direct way or in 
coordinates. Let s = $(s) = J2o<e<n ^^P^ G be a signal 
and h G Ahe a filter. Then, filtering s with h is simply the 
product 

h-s€M, (39) 

i.e., the product of the polynomials h and s modulo p. Note 
that since the result is in 7W, it is again a signal, as desired. 
Filtering, or multipUcation by h, is a Unear mapping, and so 
it has a matrix representation w.r.t. the basis b. This matrix is 
given by (j){h), where (p is the representation of A afforded by 
M. with basis b (see (11)). Thus, (39) becomes in coordinate 
form 

<j){h) • s e C". (40) 

In particular, we call the matrix ^(x) corresponding to the 
shift X the shift matrix. 

By itself, (40) does not reveal the underlying structure 
provided by the ^-module M. This structure is explicit in 
the coordinate-free representation of filtering given in (39). 

Example. In our example, let h = ho + hix G A = 
C[x]/(x^ — 1) be an arbitrary filter^'. To compute its matrix 
representation w.r.t. the basis b — (1, x) of M, we apply 
the filter to the base vectors (unit impulses) to obtain their 
impulse responses; the coefficient vectors of these responses 
are the columns of (p{h). We have h-1 — ho + hix e M and 
h - X = hox + hix"^ = hi+ hox mod (x^ — 1). Thus 



ho hi 
hi ho 



and 



h - s (j){h) ■ s. 



In particular, the shift matrix is given by 



^'Note that the following derivation does not depend on the chosen basis 
in A; in fact, we view h here as a polynomial, not as a filter expressed in 
the basis {l,x} (admittedly a subtle difference). 



20 



D. Visualization 

The visualization of the signal model A^,<1>) with $ 
in (38) is the graph with n vertices that has the shift matrix 
(^(x) as adjacency matrix (see Definition 2). In the general 
case considered in this section, this matrix has no apparent 
structure. 

The signal models that are actually used in signal process- 
ing, however, do have structure. In particular, is always 
very sparse for these models. Deriving and explaining this 
structure is one of the tasks that we solve in this paper. 

E. Spectrum and Fourier Transform 

Unless stated otherwise, we assume that p{x) is a separable 
polynomial, i.e., 

fe=0 

We set a = (ao. • • • , In words, separability means that 

p has no zeros of multiplicity larger than 1 . We will see below 
that this property ensures that the spectrum of M. consists 
exclusively of one-dimensional spectral components. 

The Fourier transform, or spectral decomposition, of the 
regular module M = C[x]/p{x) is given by the Chinese 
remainder theorem (CRT; stated in Theorem 43 in Appendix I) 
and, as filtering above, can be expressed in a direct, coordinate- 
free way, or, alternatively, using coordinates w.r.t. a given 
basis. 

In a coordinate-free form, the Fourier transform is given by 
the mapping 

A : C[x]/p{x) C[x]/{x -ao)®...® C[x]/{x - a„_i), 
s = s{x) (s(q!o), . . . , s(a„_i)). 

(41) 

Each Aik = C[a;]/(a; — afc) is of dimension 1. So the elements 
(vectors) of C[x]/{x — ak) are polynomials of degree or 
scalars c e C. Further, Mk is an ^-module, since for h = 
h{x) e A and c e Mk, 

h{x) ■ c = h{ak) ■ c mod {x — ak), 

i.e., the result is again in A4k. Since A4k is of dimension 1, 
it is irreducible. 

The scalars s{ak) in (41) are the spectral components of s. 
The mapping in (41) simultaneously projects a signal (i.e., 
polynomial) s € C[x]/p{x) into the modules C[x]/(x — ak). 
This projection is precisely the evaluation 

s{x) = s{ak) mod {x — ak). 

The set of one-dimensional irreducible submodules Mk = 
C[x]/{x — ak) is the spectrum of the signal space M. Each 
submodule A^^ is a simultaneous eigenspace to all filters (or 
linear systems) in A. The spectrum of a signal s e is the 
vector A(s) = {s{ao), s(a„_i)). 

Example. In our rurming example, the Fourier transform is 
given by 

A : C[x]/{x'^ - 1) ^ C[x]/{x - 1) e C[x]/{x + 1), 
s = s{x) ^ 

The Fourier transform of the signal s{x) = 1 -|- 3a; is (4, —2). 



F. Frequency Response 

Filtering in the regular module M = C[x]/p{x) becomes 
parallel filtering in the frequency domain, i.e., on the irre- 
ducible ^-modules Mk. Namely, let h G A be any filter and 
let s{ak) G Mk be a spectral component of a signal .s. Then 
the filtering by h of the signal given by this spectral component 
s{ak) is 

h{x) ■ s{ak) = h(ak)s(ak) mod {x - ak)- 

This shows that Mk affords the irreducible representation (/ife 
that maps 

(j)k : h = h{x) ^ h{ak)- (42) 

The collection of the 4>k{h), namely (/i(ao), . . . , /^(an-i)) 
is the frequency response of the filter h. This means that 
the fcth spectral component s{ak) of a signal s = s{x) is 
obtained in the same way as the frequency response h{ak) 
at ak, namely by evaluating polynomials. This is due to the 
simple structure of polynomial algebras; in general, this is not 
the case. In general, the spectral component is a li-dimensional 
vector (d the dimension of the spectral component Mk), while 
the frequency response of a filter h{x) at this component is 
a d X d matrix. They have the same dimensionality only for 
d=l. 

The irreducible representations (pk oi A = C[x]/p{x) 
are all different and one-dimensional, since p is separable. 
The following lemma states that these are indeed all one- 
dimensional representations. 

Lemma 3 Let A = C[x]/p{x), where p is separable with zeros 
ao, ■ . . ,an-i and let ^ : A C any representation, i.e., 
homomorphism of algebras. Then (j) is one of the (pk in (42). 

Proof: Let (p : ^ i-^ C be a representation. Then, 
because is a homomorphism of algebras (Definition 40 
in Appendix I), p{(p{x)) = (pipix)) = 0(0) = 0. Thus 
(p{x) = ak for a suitable k and (p{h{x)) = h{(p{x)) = h{ak) 
as desired. ■ 
Example. The frequency response of h G C[x]/{x^ — 1) 
is /;,(— 1)). Filtering /; • ,s in M is equivalent to the 

point- wise product {h{l)s{l),h{—l)s{—l)) in the frequency 
domain. 

G. Fourier Transform as a Matrix 

The Fourier transform A is a linear mapping, which can thus 
be expressed by a matrix T after bases are chosen. We will 
call this matrix also a Fourier transform for M. To compute 
this matrix, we choose the basis h = (po, • • • ,Pn-i), provided 
by $ in (38) for M, and the basis bk — (1) (the list containing 
the polynomial x° = 1) for each summand C[x]/ {x — ak). To 
compute JF, we apply A to the base vectors pf, the coordinate 
vectors of the result constitute the columns of !F. We compute 
A(p^) as the projection 

pe{x) = pe{ak) mod {x - ak). 

Consequently, the ^th-colunon of is the result of applying A 
to the base vector p(, namely {p{ao), • • • ,p(a„_i))^. Taken 
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together, T has the form 

= Vh,a = \peiak)]o<k,e<r. 



(43) 



We call Vb,a a polynomial transform. It is uniquely determined 
by the signal model A = A4 = C[x]p{x) and $ in (38), which 
fixes b. 

This definition coincides with the notion of a polynomial 
transform in [63], [64] and is related but different from the use 
in [65]. In [66], polynomial transforms are called polynomial 
Vandermonde matrices. 

Note that Vb.a can have entries equal to zero, but, as an 
isomorphism (as stated by the CRT), it is necessarily invertible. 

Let s = s{x) = ^sepeix) € M he a signal. Then, in 
coordinates, A in (41) becomes the matrix- vector product 



A(s) Vb,a ■ s = (s(ao), • • • , s{an-i)f e 



(44) 



In other words, (44) computes the spectrum of s w.rt. to 
the signal model {C[x]/p{x),C[x]/p{x),^) with $ defined 
in (38). 

Like we saw with filtering before, the coordinate form (44) 
of the Fourier transform alone does not readily reveal the 
underlying signal model, i.e., the algebra and module. 

The Fourier transform (in matrix form) for A4 = C[x]/p{x) 
with basis b is not uniquely determined. The degree of free- 
dom is in the choice of bases in the irreducible submodules 
C[a;]/(a; — a^) of A4 in (41). If we choose generic bases 
h = {dk), ak ^ 0, in <C[x\/{x - ak), < k < n, then 
the corresponding Fourier transform is given by the scaled 
polynomial transform 

diag(l/ao, . . . , l/o„_i) • Vb,a- 

We restate this in the following theorem. 

Theorem 4 (Fourier Transforms) The matrix is a Fourier 
transform for the regular module M = A = C[.t]/p(.t) {p 
separable), if and only if is a scaled polynomial transform 
of the form 



T = diag(l/ao, . . . , l/fln-i) • ^6,o 



(45) 



where afe 7^ and 6 is a basis for As a consequence, any 
such is invertible. 

Theorem 4 is represented by the following conomutative 
diagram (introduced in (36)): 



C[x]/p{x) 



n,, 



diaff(afe) 

In the top row, we have two times the module G^jpix) 
with the same basis 6, thus connected by the base change 
matrix given by the identity I„. We decompose the modules 
in two different ways: on the right by Vb^a^ which impUes the 
bases (1) in the irreducible modules; and, on the left, by the 



scaled polynomial transform T given in Theorem 4, which 
implies the bases (a^) in the irreducible modules. The bottom 
row shows the base change between the decomposed modules, 
which is diagonal. Connecting the top right with the bottom 
left corner in two different ways yields (45). In short, the 
diagram shows that an arbitrary Fourier transform differs from 
the unique polynomial transform only by a diagonal matrix. 

Example. We continue our previous example p{x) — {x — 
\){x + 1) and choose the basis b = (1, x) in M. The generic 
Fourier transform for M. is given by Theorem 4 as 



diag(l/ao, l/oi) 



diag(l/ao, 1/ai) • DFT2, 



(46) 



where DFT2 = [ J _J ]. In particular, DFT2 is the polynomial 
transform for A4 = C[.t]/(.t^ — 1) with basis b. 

Furthermore if s is the coordinate vector of a signal s £ M 
w.r.t. b, then 

DFT2-s= 

H. Diagonalization Properties and Convolution Theorems 

The diagonalization property of any Fourier transform T of 
the regular module M. = C[x]/p{x) is obtained as a special 
case of (21) (see also Figure 7). 

Theorem 5 (Diagonalization Properties) Let be a Fourier 
transform for the regular ^-module C[x]/p(x) with basis b 
and corresponding representation cf) of A. Then 



T ■ A-T ^ = diag(ao, . . . , a„-i), 



(47) 



if and only if ^ = for a filter /i e »4. In this case 
ak = h{ak), < fc < n, is the frequency response of h. 

In particular, diagonalizes the shift matrix <j){x). The shift 
operator x has the frequency response (ao, • • • , ctn-i)- 

Proof: Let A — (pih). Then !F(t){h)!F^^ is diagonal, 
since it is the coordinate representation of the filter h in 
the frequency domain, which is the diagonal matrix with the 
frequency response on the diagonal (all blocks on the right in 
Figure 7 are 1 x 1). 

Conversely, the set of diagonal matrices diag(ao, . . . , a„_i) 
is an n-dimensional vector space. Since is invertible, the set 
of all matrices A diagonalized by is also n-dimensional. 
Since A is of dimension n, and (j) is injective, the set of all 
matrices is a vector space of dimension n and thus the 
set of all matrices diagonalized by !F. ■ 

We also note that, using Theorem 5, we get immediately 
the characteristic polynomial, trace, and determinant for ev- 
ery matrix 4){h), since it is similar to the diagonal ma- 
trix diag(/i(ao), • • • , h{an-i)). In particular, the characteristic 
polynomial of (f){x) is p{x). 

Example. In our example, we obtain the diagonalization 
property of the DFT2, namely, for h = ho + h^x e A, 



DFT2- 



ho 
hi 



hi 
ho 



•DFT2' = diag(Ml),/i(-l)). 



The characteristic polynomial of the shift matrix ^(a;) = ^!'(0-|- 
1 - a;) = [10] is p{x)=x'^ - 1. 
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Theorem 5 provides the essential step to obtaining a con- 
volution theorem in M., i.e., a way to perform filtering in the 
spectral domain. Namely, starting from the coordinate form 
(40) of filtering, and using (47), we get for aU a e C" 



0(/i)-s = jr-ijr0(/i)jr-ijr.s 

= ^~Miag(/i(ao),...,/i(a„-i)):F-s, 



(48) 



which again shows that multiplication by 4>{h) is the same 
as multiplying by diag(/i(ao), . . . , /i(Q:n-i)), the frequency 
response, in the spectral domain. This is illustrated in the 
following conmiutative diagram: 



C[x]/p{x) 



m 



Filtering in the signal space (top row) is equivalent to point- 
wise multiplication in the spectral domain (bottom row). 
Connecting the top left with the top right corner in two 
different ways yields (48). 

Next, we derive the general form of the convolution theo- 
rem. Since, as said above, the frequency response is obtained 
in the same way as the spectrum (evaluation of polynomials at 
the dfe), we can also compute it using a Fourier transform, even 
though we did not introduce a Fourier transform for elements 
in the algebra. 

Namely, we choose a basis &' in A and let h be the 
coordinate vector of h w.r.t. h' . Then we choose a second 
arbitrary Fourier transform T' = D ■ Vv where D is 
diagonal. Using (44), 

Pfc'.a • h = • (/i(ao), . . . , /i(a„_i))^. 

Combining this equation with (48) yields the following theo- 
rem. 



Theorem 6 (Convolution Theorem) Using previous notation, 
let M. have basis h and A = M. have basis 6'. Let s ^ M. 
and h€: A. Further, let T and = D - 'Pb',a be any Fourier 
ttansforms of M. w.r.t. the bases b and h' , respectively. Then 

(l>{h) ■ S = (i'JT)-! . ((jT' . h) (JT . s)), 

where denotes the pointwise product of vectors. 

Example. We illustrate Theorem 6 using our example 
C[a;]/(a;2 - 1) with basis choosing T' = T = DFTj. 

We get the well-known convolution theorem for the DFT (of 
size 2): 



hi ho 



I. Example: Vandermonde matrix 

As an example, we consider a generic separable polynomial 
p{x) with zeros a^, < /c < n, and choose the basis b = 
(l,x, . . . in = C[a;]/p(a;) and the bases b^ = (1) 

in C[.t]/(.t — a/c). The corresponding polynomial ttansform is 
the Vandermonde matrix 

^ = 'Pb,a = [cti]o<kJ<n- 

We evaluate the associated representation cj) at the shift op- 
erator X G A. Let p{x) = J2Pi^^^ f^n = 1- Then x ■ x^ = 
x^^^ mod p{x) for < j < n — 1, and x ■ x 
J2 —PiX^ mod p{x). Thus, the shift matrix is 



n—l j,n — 



-Pn-2 
-Pn-1 



(49) 



s = DFT2"i -(DFTs h DFTa s). 



which is the transpose of the companion mattix of p. Us- 
ing (47), 

T ■ (}){x) ■ = diag(ao, • ■ • , a„_i). 

For a convolution theorem, we choose J^' = T, and = 
to get 

(^{h)-% = T-^ -{{T ■}\)Q{T ■%)). 

IV. Where are we now? 
Up to this point we have accomplished the following. 

• We gave evidence that the basic assumptions underlying 
linear signal processing make the set of filters and signals 
not only vector spaces, but an algebra and a module, 
respectively. This places SP into the context of the 
representation theory of algebras, which includes but goes 
beyond linear algebra. In particular, filtering produces the 
module structure and the Fourier transform is usually not 
thought of as a concept from linear algebra. 

• We elaborated on the correspondence, or better, equiva- 
lence, of signal processing and the representation theory 
of algebras, by providing a small dictionary that trans- 
lates between signal processing concepts and algebraic 
concepts. 

• We formally defined the notion of a signal model for a 
vector space V of signals as a triple {A^M.,^), where 
A is the chosen filter algebra, Jv[ is an ^-module of the 
same dimension as V , and $ is a bijective linear mapping 
from V onto M.. Once the signal model is defined, all 
main ingredients to signal processing are immediately 
available. Further, different models can be applied to the 
same signal space V or signal s e V, reflecting the fact 
that different transforms (such as the DFT and the DCT) 
can be used for analysis. We will see later that both the 
DFT and DCT are Fourier transforms for suitable signal 
models. 

• We identified the shift operator as generator of the filter 
algebra A and asserted that a signal model is shift- 
invariant if and only if ^ is commutative. By further 
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requiring that A consist exclusively of FIR filters — which 
is necessarily the case for finite signal models (dim M < 
oo) — commutative algebras are equivalent to polynomial 
algebras. 

• With this as a motivation, we developed signal processing 
on polynomial algebras, focusing on one variable A = 
C[x]/p{x) equivalent to one shift operator x or 1-D sig- 
nals, and to the regular case M = A. We showed which 
specific form filtering, spectrum, Fourier transform, and 
other signal processing concepts take in this case. 

In summary, our approach so far has been "top-down:" 
we considered existing concepts in signal processing and 
estabhshed their algebraic interpretation. We used repeatedly 
the example of infinite discrete-time signal processing. In fact, 
in discrete signal processing, concepts like filter, shift, and 
Fourier transform are usually thought of in the context of 
infinite discrete time. The question that may arise now is if the 
algebraic theory goes beyond this example of infinite discrete- 
time signals. The answer is yes, as we have hinted at already 
in Section I and will show in detail in the remainder of the 
paper. 

In addressing this question, and in contrast to what was 
done before, we now take a "bottom-up" perspective: we make 
assumptions only as necessary and derive signal models from 
basic principles. This is important and beneficial from several 
points of view and addresses several fundamental questions 
such as: 

• How to construct a signal model for finite signals from a 
corresponding signal model for infinite signals? 

• How to derive a signal model for discrete and finite space 
analogous to the corresponding signal models for time? 

• Why and when do boundary conditions and signal ex- 
tensions arise, e.g., why is the periodic signal extension 
important and are others possible? 

• What type of objects are the discrete trigonometric trans- 
forms, in particular, the discrete cosine transforms (DCTs) 
and the discrete sine transforms (DSTs)? 

Finally, we believe the bottom-up derivation of signal models 
to be of educational value as it explains (and visuahzes) the 
models and provides a recipe on how to possibly construct 
new models for different applications. 

The remaining part of this paper uses the algebraic concepts 
introduced before. The reader may wish to revisit frequently 
the previous sections, connecting the following concrete ex- 
amples to the previous, more theoretical discussion. 

V. Modeling Time: The z-Transform 

Let s = (sn I n G Z) be a discrete signal. The z-transform 
of s, in standard notation, is given by the mapping 



$: Sh^ s = S{z) = '^SnZ 



(50) 



In Section II-C we showed that the ^-transform is the Unear 
mapping of a signal model in the sense of Definition 1 
for the vector space £^(Z) of finite energy sequences. The 
corresponding algebra is usually chosen as the set of all 
Laurent series with coefficient sequences in 



Naturally, the following fundamental question arises: Why 
is $ the appropriate linear mapping for the signal model for 
discrete time, or, equivalently, why is discrete time filtering 
defined the way it is? We give an answer to this question 
by deriving the ^-transform from basic principles. In other 
words, we identify the assumptions that have to be made to 
obtain the ^-transform. This knowledge will then enable us, 
for example, to derive the space-analogue of the ^-transform, 
the C-transform in Section VII. In fact, since infinite discrete 
time is standard in classical signal processing theory, the whole 
purpose of this section is to identify the recipe for constructing 
a signal model. 

The section is structured as follows. First, we build the 
signal model in the sense of Definition 1 in three steps: 1) 
definition of the shift; 2) hnear extension; and 3) reaUzation. 
Second, after the signal model is determined, we provide the 
associated spectrum and Fourier transform. And third, we give 
a visualization of the signal model in the sense of Definition 2. 
The derivation and analysis of the discrete space models and 
discrete generic next neighbor model in Sections VII, X, and 
XII will follow the same steps. 

For notational convenience, we set in the foUowing^^ x = 



A. Building the Signal Model 

Definition of the shift. Following Kalman [16], when 
considering time, we need two ingredients: time marks tn and 
a shift operator q. 

The time marks are symbolic independent variables tn,n G 
Z; tn is associated to "time n." Using these time marks, we 
can write every signal s = (s„)„gz € as the symbolic 
sum s = J2ne2,^ntn- The set of all these sums is again 
a vector space isomorphic to C^. In other words, no new 
structure was introduced in the signal space. At this point 
there is no interaction between time marks at different times 
n, in particular no notion of past and future, nor any notion 
of equidistance between consecutive time marks. 

To address this problem, we introduce the shift operator q 
and the shift operation o by 



time model: g o t„ = tn+i 



(51) 



for n e Z. Figure 1 1 shows a graphical representation of the 
time shift. 



qo 



qo 



tn-l 

Fig. 11. The time shift q o t„. 



tn 



+1 



The operator q naturally expresses a direction from past to 
future and the equidistance of the time marks 

^^Note that the choice of instead of z in the definition (50) is a 
convention, not a mathematical necessity; choosing z leads to equivalent 
properties and an equivalent theory for the z-transform. In fact, the choice of 
in signal processing is in contrast to the previous mathematical work on 
Laurent series. The reason may be the fact that the shift operator z~-^ causes a 
delay of the signal. However, we will see that, equivalently, this shift operator 
advances what we call the time marks. 
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Next, we extend the operator domain from a single shift 
operator q to k-fold shift operators qk, defined by 

Qk<>tn = tn+k- 

Clearly, qk = q^. 

At this point of the construction, working only with f„ and 
qk, there is no notion of linearity. 

Linear extension. Since we are interested in obtaining a 
linear signal model, we carry out two extensions: 1) we extend 
the operation o of the shift operator q from the set of the i„ 
to the set of all formal sums ^ Sntn, by requiring linearity, 
i.e., q o s = '^s^iq o and 2) we extend the operating 
set from the set of fc-fold shift operators q'' to the set of all 
formal sums ^hkq'^- The first set will become the module 
of signals, while the second set will become the algebra of 
filters. 

In other words, starting with (51), we first linearly extend 
the operation o to the vector space A4 = {s = J2 Sntn}, and 
then we extend the operator domain to obtain A = {h = 
J2f^k<l'^}- Because the series in either set have an infinite 
number of terms, we need to make sure that filtering is well 
defined: the multiplication of an infinite series — the signal 
s e A4 — ^by another infinite series — the filter h G A — has 
to exist. We consider this in the next step, called "realization." 

Realization. To obtain the signal model, we first consider 
the "realization" of the abstract model, which replaces the 
abstract objects t„ and q and the operation o by objects we 
can compute with. To this end, we choose a variable x and set 
q = X, and <> = •, the ordinary multiplication of series. Then 
(51) becomes 

tn+l = X-tn- (52) 

This two-term recurrence, when started with to = 1, has the 
unique solution 

t„ = x". (53) 

In other words, the realization is essentially (up to a common 
scaling factor for all a;") unique. 

As a result, we obtain A4 = {s = J2 -^na^"} and A = 
{h = Yli^kx'^}- Since the series are infinite, we have to 
ensure convergence as part of the realization; namely, that 
filtering, the operation of ^ on A^, is well-defined. This is 
achieved, for example, by requiring s G i?^(Z) and li e 
as explained in Section II-B. Now A becomes the filter space 
and M becomes the signal space. 

Table IV shows the correspondence between the abstract 
and the realized concepts. 

At this point it seems that the only reason for doing a 
"realization" is to handle convergence issues. We will see 
later in the derivation of the space model that the realization 
may not be unique and thus may lead to different final signal 
models. 

Signal model: z-transform. Now we can formally define 

the signal model {A,M., $) for discrete infinite time. The filter 
algebra A is the set of Laurent series h with coefficients h e 
^^(Z); the signal module is the set of Laurent series s with 
s e ^^(Z). The third component of the signal model is the 



TABLE IV 

Realization of the abstract time model. 



concept 


abstract 


realized 


shift operator 


<? 


X 


shift operation 


o 




time mark 






fe-fold shift operator 


qk = 9* 


x'^ 


shift 


qOtn = tn + l 




signal 




SnX^ 


filter 


T.hkq'' 


Yjhkx'^ 



bijective linear mapping which maps signals from the space 
V = e{I.) m\oM: 

is the z-transform. 

Note that signals and filters are conceptually different (as 
pointed out several times before) but look the same (both are 
Laurent series in x) because the realization maps both and 

tn to X"'. 

B. Spectrum and Fourier Transform: DTFT 

We discussed the spectrum and the Fourier transform for 
the z-transform already in Section II-B, so we are brief here. 

The ^-module M = $(£^(Z)) decomposes into continu- 
ously many one-dimensional irreducible modules A4u;, u) € 
W = (— TT, tt], which constitute the spectrum of A4. Each 
spectral component Ad^i is spanned by in (23). As we 
stated before, the are not contained in $(£^(Z)), but only 
in $(^°"(Z)), so the Muj are not submodules of M. Choosing 
E^ as basis in A4^., the Fourier transform A (in the form (16)) 
is called the discrete-time Fourier transform (DTFT) and is 
given by 

s = S{x) ^ {S{e-^'^))^ew = w ^ 6'(e-^'^). 

Note that we have to write S{e^^'^) since our variable is x 
instead of z~^. The spectrum A(s) of a signal can be viewed 
as a (complex L^-) function on the unit circle. 

Every irreducible module A4^. affords a one-dimensional 
irreducible representation (j)^. Namely, if h = H{x) £ A is a 
filter, then 

H{x)EUx) = Hie-^E^x), 
which shows that 

cj>UH{x)) = H{e-n- 
The frequency response of ft = H{x) is the collection, for 

U> e (— TT, tt], 

(ft))c. = {H{e-n)u> =co^ H{e-n- 

Filtering h ■ s = H(x)S{x) in the time-domain becomes 
the parallel multiplication H{e~^'^)S{e~^'^), w & W by the 
frequency response in the frequency domain. 
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C. Visualization 

Because of the simple shift operation associated with the z- 
transform, the structure of M. can be conveniently represented 
as a graph, shown before in Figure 10, which expresses the 
shift, i.e., the multiplication by x in the chosen realization. 
Formally, the graph is obtained by using our Definition 2 of 
the visualization of a signal model. We have one shift operator 
X, and, if 4> is the representation afforded by M with the time 
marks x^ as basis, then (/)(x) is the adjacency matrix of the 
graph in Figure 10. 

The bullets in Figure 10 represent the time marks to which 
the signal values are associated via the signal model, i.e., via 
the ^;-transform (50). If the signal space is considered as a 
vector space these bullets become the basis, but are uncon- 
nected. The connecting edges arise from the shift operation, 
which makes the signal space a module. 

Figure 10 is helpful to obtain an intuitive understanding of 
the module structure and is clearly familiar to most readers. 
We will show similar graphs for the finite z-transform and 
their space model analogues below. Comparing these graphs 
is the a simple way to intuitively understand the differences 
between the associated signal models. 

VI. Finite z-Transform and DFT 

In real applications, usually only a finite subsequence s = 
(so, . . . , Sn-i) is available, not the entire (sampled) sequence 
[sk \ k € Z). Thus, the question is how to develop a finite 
version of the 2;-transform, i.e., with a finite number of terms. 
We will see that the problem, when restricted to a finite 
sequence, is in preserving the module property to have access 
to filters and a notion of spectrum and Fourier transform. 
It turns out that to obtain the module structure, we need 
to introduce boundary conditions (b.c.'s). The b.c.'s in turn 
determine a polynomial algebra and the associated module. 
This gives a "bottom-up" justification for polynomial algebras 
in addition to the motivation presented in Section II-D, which 
identified them as those algebras providing finite shift-invariant 
signal models. The bijective linear map $ in the signal model 
we obtain, i.e., the associated finite ^-transform, maps signals 
s G C" into this module. The chosen b.c.'s determine the 
signal extension, and, by requiring a simple signal extension 
(in a sense defined below), we naturally obtain the DFT (and 
its variants) as Fourier transform for the associated module. 

Interestingly, the concept of a finite z-transform in the sense 
defined below is not used in signal processing, even though it 
is the precise finite analogue of the standard z-transform. Fur- 
ther, it explains the DFT and the role of boundary conditions 
and signal extension, and facilitates the derivation of various 
DFT algorithms, just to name a few benefits. 

For our investigation, we first need a formal notion of signal 
extension, which we define now. 

Definition 7 (Signal Extension) Let s = (sfc | fc S /) be a 
signal given on an index set 7 c Z. A (linear) signal extension 
of s is the sequence of linear combinations (only finitely many 
summands are nonzero) 

Sk^^l3k,iSi, foik^I. 



The signal extension is called monomial, if, for each k, the 
sum has only one summand. 

In other words, in a monomial signal extension, every signal 
value outside the signal scope is assumed to be a multiple of 
a signal value inside the signal scope. 

A. Building the Signal Model 

Shift, linear extension, realization. To construct a finite 
time signal model, we follow the exact same steps as in 
Section V-A. We start with the time marks, of which we have 
now only a finite set to, ... , in-i- As before, we consider the 
shift operator g, the operation o and the realization of the shift 
operator by the variable x. However, as we will see next, this 
leads to one important difference. 

Let s = (so) • • • , Sn-i) S C" be a sampled signal. To 
realize time, we could attempt to realize as before the two 
step recursion (52) and to define the "finite" ^;-transform, by 
mapping s to the polynomial of degree less than n 

s = s{x) = Skx''. 

0<k<n 

Proceeding with the definition of the signal model, we attempt 
to identify the module and the algebra. Clearly, the set C„ [/;] 
of the polynomials s{x) (of degree less than n) is a vector 
space with the natural basis b= (x", . . . , x"~^). The problem, 
however, arises from the operation of the (realized) time shift 
operator x: the set of polynomials of degree less than n is not 
closed under multiplication by x. More precisely, the root of 
the problem is 

a;-a;"-^ =x" ^C„[a;], (54) 
and, if non-causal filters are considered, 

x-'^ ■ x^ = x-'^ ^ Cn[x]. (55) 

Thus, the time shift as has been defined is not a valid 
operation on C„[a;], which implies that we cannot define 
filtering in C„[a;], or, algebraically, C„[a;] is not a module. 
Without filtering, there is also no notion of spectrum or Fourier 
transform. To resolve this we need to take care of the problems 
raised by (54) and (55), which we do by introducing boundary 
conditions. 

Boundary condition and signal extension. To remedy the 
first problem (54), we have to make sure that x" can be 
expressed as a polynomial of degree n — 1. This is achieved 
by introducing an equation 

a;" = r{x) = J2 ^kx'', or a;" - r{x) = 0. (56) 

0<fe<n 

This equation is equivalent to the right boundary condition 

0<k<n 

As a consequence of (56), we get the series of equations 

x''{x"- - r{x)) = a;'= • = 0, A; > 0. 

Thus, the boundary condition a;" = r{x) determines the 
entire right signal extension that is obtained by reducing 3;*^+" 
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modulo (a;" — r{x)) to a polynomial rk{x) of degree less than 
n, i.e., 

= rfc(x) mod (.x" - r{x)). (57) 

Algebraically, the boundary condition replaces the vector 
space Cn[x] by the vector space M = C[x]/{x" — r{x)), 
which is of the same dimension, but closed under multiplica- 
tion by the time shift operator x and is thus a module. The 
corresponding algebra A, generated by x, is identical to A4. 
The remaining question to consider is (55). There are two 
cases. 

Case 1: x\r{x). Then also x\{x^ — r{x)), and thus x (the 
shift operator) is not invertible^^ in ^ = C[x]/(a;" — r(x)) 
and (55) does not need to be considered: the signal has no left 
b.c, since "the past" is not accessible without an invertible x. 

Case 2: x /(r{x). Then, from (56), we get 

Po 

which is the left boundary condition. Similar to above, the left 
signal extension can be determined by multiplying by x^'' and 
reducing modulo a;" — r{x). Thus, the signal extension in both 
directions is determined by one equation (56), which provides 
the left and the right b.c: 

b.c. => right and left signal extension. 

By assuming the generic boundary condition a;" = r{x), 
we obtain a valid signal model. However, the corresponding 
signal extension (57) has in general no simple structure. To 
obtain a module that is reasonable for applications, we thus 
require 

• the shift operator a; e ^ to be invertible; and 

• the signal extension to be monomial (see Definition 7). 
A monomial signal extension is the condition that leads to the 
signal model for the DFT in the finite time case and for the 
16 DCTs and DSTs in the finite space case. In Section IX, 
we will slightly relax this condition and obtain a new class of 
transforms. 

We can now explicitly determine the polynomials x" — r{x) 
that satisfy the above two conditions. 

Lemma 8 The boundary condition x" = r(x) makes A = 
C[a;]/(a;" — r(a;)) an algebra in which x is invertible and 
determines a monomial signal extension in Ai = A, if 
and only if the polynomial r{x) is a nonzero constant, i.e., 
r(x) = a 7^ 0. The signal extension in this case is given by 
_ Qfc2 2,fci^ where k £ Z is expressed as fc = fci + k2'n, 
with < fci < n. 

Proof: Let r{x) = a, a ^ 0, and let k £ Z. We 
write k = ki + k2n, with < fci < n, and thus x'^ = 
ijk2j,ki j^j-jj _ which is a monomial signal extension. 
Conversely, let x" = r{x) determine a monomial signal 
extension. This implies x" = ax^, for some < i < n. Since 

^^A polynomial q{x) e C[x]/p{x) is invertible if and only if 
gcd{q{x) , p{x)) = 1, since in this case there are polynomials r{x),s{x) 
such that 1 = s(x)q(x) + r{x)p{x), which implies that s{x) = 
q{x)~^ mod p{x). 
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Fig. 12. The signal extension of C[x]/(x" — a). 

X is by assumption invertible modulo x" — ax^, it follows 
£ = and a j^O as desired. ■ 

It is instructive to graphically display the signal extension 
associated to = C[x]/(x" — a). There are several ways 
to do this. We choose to display it in a virtual coordinate 
system. The x-axis carries the time marks x% i e Z, and the 
y-axis carries the basis .t", . . . , x"^^ of A4. For every i e Z, 
we express x' in the basis and enter the coefficients in the 
graph. Figure 12 shows the result (with the y-axis coordinates 
omitted). Within the signal scope (shown by the bold line) < 
i < n, we have the identity (since x* = 1 • x*); for n < i < 2n 
the line is scaled by a (since x* = ax*~" mod (x" — a)), and 
so on. The bullets mark starting points (at multiples of n) and 
end points of the lines to enhance the presentation. 

Using Lemma 8 we observe that the signal extension 
corresponding to the b.c. x" = a is eventually periodic, if 
and only if a is an arbitrary mth root of unity. The period in 
this case has length mn. 

Signal model: finite z-transform. In summary, the signal 
space so obtained is the regular module A4 = C[x]/(x" — o) 
with algebra of filters A = A4. The final component of the 
associated signal model for F = C" is the finite z-transform 

$ : s 1-^ ^ SfeX*" e M 

0<k<n 

that endows the signal s G C" with the structure of the 
module M. The associated algebra A = M of filters has 
the same basis b = (1, x, . . . , x"~^) as A4; in the basis 
polynomials (or impulses) x'^ represent time marks; in A they 
represent (fc-fold) time shift operators. 

Note that we restricted this definition of a finite 2-transform 
to the cases of monomial signal extensions only, since these 
are the only ones we consider in this paper. Of course, the 
definition could be generalized to any p(x) = x" — r(x). 

B. Spectrum and Fourier transform: DFT and variants 

Using the general results from Section III, we now derive 
the spectrum and the Fourier transform for the signal model 
finite z-transform. In other words, we consider the regular 
module M. = A = C[x]/(x" — a) with the same basis h = 
(x°, x^, . . . x"~^) chosen in M. and A. We use the notation 
i = in this section. 

Since a ^ 0, p{x) = x" — a is separable. Let a = \a\e''^ in 
polar coordinates. The zeros au of x" — a are given by 

1 (i^-2fc7r)j L Hi I. 

ak = \a\ne n =|a|nen(jj„, (58) 

where a;„ = e^^'^-'/". Thus, spectrum and Fourier transform 
of C[x]/ (x" — a) are given by 

A: C[x]/(x"-a) ^ ©o<fe<„ C[x]/(x - a,) 
s = s{x) (s(ao),...,s(a„-i)). 
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In matrix form, the polynomial transform corresponding to 
A is given by 



where 



n,a = DFT„ • diago<^<„ (|a| « e « ) , (59) 



DFT„ = K ]o<fc,£< 



is the (standard) discrete Fourier transform. Using Theorem 4, 
all Fourier transforms are given by 



(60) 



where D is any invertible diagonal matrix D, which is 
determined by the basis chosen in the spectrum. 

This derivation shows the origin of the DFT in signal 
processing terms, namely, the DFT arises by constructing a 
finite time model under the assumption of a monomial signal 
extension. The standard DFT (without scaling factors) arises 
from the special case a = 1 and = I„, which implies that 
the DFT„ is a polynomial transform. 

Generalized DFTs. The class of DFTs (60) also includes as 
special cases what have sometimes been called "generalized 
Fourier transforms" (see [67], [68], [69]). We prefer "gen- 
eralized DFTs," since, as we have explained in Section II- 
B, the concept of Fourier transform is far more general, 
encompassing decompositions of arbitrary modules. 

The generalized DFTs are matrices of the form 

where c,d S R. We briefly investigate the 4 special cases 
given by c, d € {0, 1/2}, which in [69] are called DFTs of 
types \-A, written as DFT-1, . . . , DFT-4. Namely, 

DFT-1„ = Tofl = DFT„, 

DFT-2„ = To,i/2 = diago<fc<„(u;^/2) dft„, (61) 

DFT-3„ = 0, (62) 

DFT-4„ = jr,/2,i/2 = diago<fc<„(a;i'=+^^/')DFT-3„. (63) 

We identify the signal models for which these transforms 
are Fourier transforms, by comparing their definitions to 
(60) and (59). The DFT-1„ = DFT„ is, as said above, a 
polynomial transform for C[x]/{x^ - 1). The DFT-2„ in (61) 
is also a Fourier transform, but not the polynomial transform, 
for C[a;]/(a;" - 1). The DFT-3„ in (62) is the polynomial 
transform for C[x]/(x" -|- 1), since Wn"*"^^^, < A; < n, are 
precisely the zeros of x" + 1. Finally, the DFT-4„ in (63) 
is also a Fourier transform, but not the polynomial transform, 
for C[a;]/(x" -|- 1). This means, these DFTs cover the two 
important cases of boundary conditions = a = ±1. 

The DFT„ = DFT-1„ is the matrix containing the 
evaluation of the polynomials at the roots of unity uj^. 
Similarly, the matrix DFT-2„ consists of the evaluations of 
the "fractional" polynomials x^"*"^/^ at the w^. Thus, in a 
sense, and leaving our framework of polynomial algebras, 
DFT-2„ could be viewed as a Fourier transform for the A- 
module M spanned by the basis {x^/^, x^/^, . . . , 
and with periodic boundary condition a;"+^/^ = x^/^, and 
A = C[x]/(x" — 1) as operating algebra. In other words. 



DFT-2„ can be viewed as a "shifted" version of DFT„. This 
shifting, as we showed above, does not change the underlying 
signal model. 

A similar relationship exists between DFT-3„ and DFT-4„. 
Other boundary conditions and effect on spectrum. At 

this point it is instructive to investigate what problems arise 
if we slightly relax the conditions in Lemma 8 by dropping 
the requirement that the shift operator x is invertible in the 
constructed algebra. In particular, this includes, as we show, 
the zero extension and the constant extension. 

The proof of Lemma 8 shows that a monomial signal 
extension requires a b.c. of the form x" = ax^, i.e., p{x) = 
x" — ax^. Conversely, each such b.c. determines a monomial 
signal extension. A simple choice is a = yielding the 
regular module C[x]/x", which realizes a right zero extension 
(x" = implies x""*"*^ = for k > 0). There is no concept of a 
left extension, since the shift operator x is not invertible in A. 
The problem of this model is the spectrum: C[x]/x" cannot be 
decomposed by the Chinese Remainder Theorem (CRT), or, in 
other words, the model is not appropriate for spectral analysis. 
If (j) is the representation afforded by M, then this can also be 
seen from the shift matrix 4'{x), which is the (lower) Jordan 
block (a special case of (49)) 



<t>{x) = 





1 



1 



As a Jordan block, this matrix cannot be block diagonaUzed 
any further. 

Another simple choice is the symmetric b.c. x" = x"~^, 
i.e., p{x) = x" — x"~^. This choice implies a constant right 
signal extension, since x" = x"~^ impUes x""*"*^ = x"~^ for 
all fc > 0. In this case, the CRT yields 

C[x]/(x" - x"-i) ^ C[x]/(x - 1) ® C[x]/x"-\ 

and the rightmost module, of dimension n — 1, is again 
indecomposable, making spectral analysis trivial. We will see 
later, that a symmetric signal extension in a discrete finite 
space model leads to a monomial signal extension and to a 
separable polynomial p, and thus provides a useful model for 
signal processing. 

As a final example, we consider the generic right b.c. x" = 
r(x), such that p{x) = x" — r(x) is separable. This leads to a 
generic regular module C[x]/p{x) with basis (1. .t, . . . , x'"-^^). 
As we have seen in Section 111-1, the (polynomial) Fourier 
transform in this case is a Vandermonde matrix and the shift 
matrix (p{x) is the companion matrix (49). In other words, 
Vandermonde matrices are precisely the (polynomial) Fourier 
transforms for (separable) finite time models. 

The above discussion shows that the choice of boundary 
condition affects both, the signal extension and the notion of 
spectrum. Models that have "good" properties with respect to 
both are useful in signal processing. 

Signal extensions that are not linear (for example second or- 
der polynomial extensions) are sometimes considered in signal 
processing. These do not produce signal models (in particular. 
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a 




Fig. 13. Visualization of the finite discrete time model in the case of a 
monomial signal extension A = M. = C[x\/(x^ — a). 



All entries in (t>{h) above the main diagonal are due to the 
signal extension. Matrices with this structure are sometimes 
called a-circulant (e.g., [70J). For a = 1, (/>(/i) is an ordinary 
circulant matrix, which is diagonalized by T = DFT„. The 
structure of the matrix (j){h) reflects the choice of basis b in 
the algebra A, namely, (j){h) is a sum of weighted A;-fold time 
shifts as seen from (65). 



filtering becomes non-linear) and are thus not covered by our 
theory. 

C. Visualization 

The structure of the signal model given by the finite z- 
transform is visualized by the graph in Figure 13. Following 
Definition 2, the adjacency matrix of this graph is the shift 
matrix (j){x) ((/> is the representation afforded by M.): 



cp{x) = 





1 







(64) 



which is again a special case of (49). The boundary condition 
= a is represented by the weighted edge from a;"~^ to x°. 



D. Diagonalization Properties 

To determine the diagonalization properties of ,F in (60), we 
need to calculate the coordinate form (j)(h) of filters h & A 
(see Theorem 5). 

We consider first h = x"-. To compute 4>(K), we determine 
how h operates on the basis h oi M.. We get, for < A; < n. 



X ■ X = 



Thus, ^(/i) has the entry 1 at positions (i, j) with i — j = £, 
the entry a at positions with i — j = i — n, and zero 
entries else (0 < i,j < n). The special case £ — 1 yields the 
shift matrix (64) for a = 1, also called cycUc shift. In the 
general case h = '^ h^x^, 

^(h) = <t>{J2 hix') = hfA^') (65) 
is the generic matrix diagonahzed by in (60) and given by 



Further, by Theorem 5, 

T ■(t){h)-T~^ = 





ahn-1 


ahn-2 ■ ■ ■ 


ahi 


hi 


ho 


ahn-1 ■ ■ ■ 


ah2 


h2 


hi 


ho 


ahn-1 


n— 1 




h2 hi 


ho 



■ (66) 



diag(/i(Q;o), . . .,h{an-i)), 



where the ak are the zeros of x" — a given by (58). The most 
prominent special case is obtained for h = x and a = 1, and 
shows that the DFT diagonalizes the cyclic shift. 



E. Convolution Theorem 

Filtering in the signal space M is the multiplication of two 
polynomials s G M (the signal) and h G A (the filter) modulo 
— a. Using Theorem 6, we obtain the following known 
convolution theorem, where !F is any matrix in (60): 

h ■ s mod (x" - a) <;=^ • s 

where, as usual, h, s are the coefficient vectors of h, s with 
respect to the conmion basis b of A and M., respectively. 

F. Unitary Transform 

The matrix DFT„ diagonaUzes the matrix ^(x), which is 
unitary and has pairwise distinct eigenvalues. Thus, a diagonal 
matrix D exists such that D DFT„ is unitary. It is well-known 
(and easily computable) that D = l/y/nln- Since the scaling 
is from the left, and using Theorem 4, it follows that the 
unitary DFT is also a Fourier transform for C[a;]/(x" — 1). 
Similarly, for all generahzed DFTs of types t G {1,2,3,4} 
(see Section VI-B), 

^DFT-t„ (67) 

V"- 

is a unitary matrix, and is a Fourier transform for the same 
signal model as its non-unitary counterpart. 

G. Real Signal Model: RDFTs and DHTs 

One question that arises is how to include the real versions 
of the DFT, in particular, the real discrete Fourier transform 
(RDFT) and the discrete Hartley transform (DHT), in the 
algebraic theory of signal processing. In other words, for 
which signal models are the RDFT and the DHT Fourier 
transforms? It turns out that different signal models can be 
associated to these transforms, which give rise to different 
interpretations. In this paper we discuss two interpretations. 
The first interpretation, in this section, identifies these trans- 
forms according to their name as real DFTs, i.e., as Fourier 
transforms for the real analogue of the finite time model. The 
second interpretation identifies them as Fourier transforms for 
a particular choice of finite space model and is discussed in 
Section XI. 

The algebraic interpretation of the DHT as real DFT in 
this section is equivalent to recognizing the DHT as a special 
case of an ADFT (algebraic discrete Fourier transform), a 
general concept introduced in [8], [71] and rediscovered (using 
a different name) in [72] to map the DFT and its algorithms 
into a basefield smaller than C, i.e., a basefield in which the 
nth roots of unity are not available. Using this method, DHT 
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algorithms are readily obtained from their DFT counterparts 
[73], [72]. 

Real signal model. To derive the real DFTs, we start with 
the finite time model (see Section VI-A), but restrict it to real 
signals s e K": 

SH-» ^ Skx'' e M=A = R[x]/{x'' -1). (68) 

0<fe<n 

This model has the same visualization as its complex coun- 
terpart and thus imposes the same structure. This imphes that 
filtering is defined as in the complex case but restricted to real 
filters. 

Spectrum and Fourier transform. The difference arises 
when computing the spectrum. Since only real numbers are 
available, and the roots of x" — 1 are complex, M cannot 
be decomposed into one-dimensional irreducible modules. In 
algebraic terms, M is not a splitting field for the ^-module A4. 
Over R, the irreducible factors of x" — 1 are polynomials of 
degree 1 or 2. Namely, if are conjugated complex 

roots of a;" — 1, i.e., k ^ Q,n/2, then 

[x - Lo^-^){x - w^"'') = - cos(2fc7r/n)a; + 1 

is irreducible over M. In other words, if 

C[x]/{x - a;°),C[x]/(x - a;^), . . . ,C[x]/(x - 

is the spectrum of the complex finite time model, then the 
spectrum of the real finite time model consists of one or two 
irreducible modules of dimension 1: 

K[x]/(x-l) and M[x]/(x + l) = R[x]/(x-</2), (n even), 
and the remaining spectral components are of dimension 2: 

R[x]/(a;^ - 2cos(2fc7r/n) + 1), 1 < < n/2. 

Each of these spectral components affords a two-dimensional 
representation 0^; if /i is a filter, then 4>k{h) is the frequency 
response at frequency k. The real spectrum is obtained from 
the complex spectrum by fusing every complex spectral com- 
ponent with its conjugate counterpart. In the two-dimensional 
spectral components any real basis can be chosen. Thus, any 
real matrix of the form 



T = X ■ DFT„ 
with an invertible matrix X of the x-shaped form 



(69) 



X 







(70) 



is a Fourier transform for the real finite time model. The matrix 
X consists of one 1x1 block at entry with row and column 
indices and one 1x1 block at entry with row and colunm 
indices n/2 (if n is even). The remaining 2x2 blocks in (70) 
occur at index sets {k, n — k) x {k,n— k), for 1 < < n/2; 
fc, n — fc are the indices of conjugate pairs of complex spectral 
components of C[x]/(x" — 1). Examples of such transforms 



include the real discrete Fourier transform (RDFT) and the 
discrete Hartley transform (DHT) [74], defined respectively 
by 



RDFT„ = [ru]o<k,i<n, 



with 



and 



ru = 



■ sm 



2TTke 



DHT„ = [cos 2M2L 

2fcfal 



< fc < n/2, 
n/2 < k <n. 



sm ——\o<k,e,n 



Compare this to the DFT, defined as 

DFT„ = [cos ^ - j sin ^]o<kAn- 

The RDFT replaces two conjugate base vectors with indices 
k, n—k in the DFT with real and imaginary part as (j = \/— T) 



(71) 



a\ _ 1 r 1 l] /a + bj 
b) ~ 2 [-j j\ \a-bj 

which determines the matrix X = X'^^^'^. Similarly, the DHT 
replaces two conjugate base vectors by the difference and sum 
of their real and imaginary part as 



a — b 
a + b 



1+J 

1-i 



i + i 



bj 



(72) 



which again determines X = X^^'^. 

We can now represent the set of all Fourier transforms for 
the real finite time model more conveniently by the set of all 
matrices 

:F = X-RDFT„, (73) 

where X is any real, invertible matrix of the shape in (70). 
For example. 



DHT„ = X^'^'iX 



DHT/ vRDFTn-1 



RDFT„ , 



The reader should compare (73) to the set of all Fourier 
transforms in Theorem 4. There, the spectrum consisted only 
of one-dimensional spectral components; thus, the degree of 
freedom in choosing a basis in the spectrum yielded a diagonal 
matrix. Here, two-dimensional modules occur; thus, the degree 
of freedom in choosing bases is larger and leads to the x- 
shaped matrix X. 

Both transforms, the RDFT and the DHT, are special among 
the class of all possible real DFTs. The RDFT appears to have 
the lowest arithmetic complexity^"* and the DHT is uniquely 
determined (up to a diagonal matrix D with diagonal entries 
1 or -1) by being equal to its inverse. 

In Section VI-B we discussed 4 types of DFTs and identified 
them as Fourier transforms (scaled or unsealed polynomial 
transforms) for C[x]/(x" — 1) and C[x]/{x" + !)■ Using the 
above methods (71) and (72), each of these 4 types has a corre- 
sponding "RDFT" and "DHT," which we denote accordingly 
with RDFT-1, . . . , RDFT-4, and with DHT-1, . . . , DHT-4, 
respectively. By construction, the RDFT and the DHT of 
type t, t G {1,2,3,4}, are Fourier transforms for the real 

^We do not have a proof. The assertion is based on the best known 
algorithms. 
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counterpart of the complex signal model for the DFT of type 
t. For completeness, we provide the definitions for types 2-4: 



RBFT-t=[rk,e]o<k,e<n, 



with 



cos 

^IM£±1M, 0<fc<(n-l)/2, 
t-2: r-fe,- j_^.^2^MaiM, (n-l)/2<fc<n. 



t=3: rke 



cos 



2iT(k+l/2)i 



0<k< (n- l)/2, 



-sm^Jlih±im^ (n-l)/2<fc<n. 

cos ^-(^+V2)«+i/2) ^ 0<fc<n/2-l, 
, n/2 - 1 < A; < n. 



t^: ''fe^ - I _ gjjj 27r(fc+l/2)(£+l/2) 

And, 



DHT-2„ = [casM£±lZ2)£]„<,^,^„, 
DHT-3„ = [cas^^^±^]o<fe,^,„, 
DHT-4„ = \cas^i^±^^^^^^]o<,,e,n. 
Further, for f e {1, 2, 3, 4}, we have relations of the form 

RDFT-t = X(*)DFT-t„, 
DHT-t = X(*)'DFT-t„, 



(74) 



where X^*), and are of the form (70) for t = 1,2, and 

of the form 



X = 



(75) 



for t = 3,A. 

The four transforms DHT-t were introduced (in their or- 
thogonal form) in [75], [76], [77], [78], where they were 
called discrete W transforms (DWTs) of type 1-4. Our above 
suggestion to renaming these transforms to DHTs of type 1-4 
is motivated by 1) the name DHT (for type 1) is much more 
conmionly used than DWT, and the types 2-4 are just variants; 
and 2) even though the DHT and the DWT were introduced 
at the about the same time ([74] and [78]), the continuous 
counterpart was introduced by Hartley already in 1942 [79]. 

Diagonalization properties. The above discussion gives 
immediately the "diagonalization" properties of the RDFT and 
DHT. We use double quotes, since these properties are not 
actually a diagonalization. Jf h G A = M[a;]/(a;" — 1) is any 
filter, then (j){h) is a real circulant matrix, i.e, of the form (66) 
with a = 1. Then 



RDFT„ (j>{h) RDFT"^ = X, 



(76) 



where X is real and of the form (70). The same holds, if 
we replace RDFT by any other real DFT including the DHT. 
Of course, the RDFT and DHT have also true diagonalization 
properties (as every invertible matrix), but they do not arise 



from their interpretation in this section, but from their different 
interpretation in Section XI. 

Further, (76) generalizes to the RDFTs and DHTs of types 
2-4. For type 3 and 4, X in (76) has the form (75). 

Similarly, convolution theorems can be derived. Also, the 
above discussion can be easily generalized to every real signal 
model for the case of a generic monomial signal extension 
M[a;]/(a;" - a), a € M. 

Orthogonal transform. The orthogonal version of the 
RDFTs and DHTs of types 1-4 follows directly from (67) 
and (74), namely, for f e {1, 2, 3, 4}, 



-D ■ RDFT-t„ 

n 



and 



- • DHT-t„ 

n 



are orthogonal, where 



D= i 



{ diag(V2, 1, . . . , 1, V2, 1, . . . , 1), i = 1, 2, n even, 

diag(-\/2, 1, . . . , 1), t = 1,2, n odd, 

In, t = 3,4, n even 

,diag(l,...,l,\/2,l,...,l), t = 3,4, n odd. 



H. Rational Signal Model 

At this point it is interesting to extend the above discussion 
of real DFTs by further reducing the base field from R to the 
field of rational numbers Q. 

Rational signal model. The hnear mapping for the signal 
model is now 



0<fe<n 



Skx'' eM=A = Q[a;]/(a;" - 1). (77) 



As in the real case before, the model imposes the same 
structure (visuahzation, notion of filtering) as its complex 
counterpart with the restriction that signals and filters have 
coefficients in Q. 

Spectrum and Fourier transform. Reducing the basefield 
in the real case above had the effect of fusing spectral com- 
ponents. This effect is even more pronounced in the rational 
case as we show next. 

We restrict ourselves to a 2-power size n = 2*^. Then, a;" — 1 
decomposes over Q into irreducible factors as 

x'^'' - l = {x- l){x + l){x'^ + 1)--- {x^"'^ + 1). 

These factors determine the spectrum of the rational signal 
model namely, the spectral components are 

C[x]/{x - 1) and C[x]/{x'^* + 1), < i < k. 

Clearly, there is now a large degree of freedom in choosing 
bases in the spectral components, i.e., in defining a Fourier 
transform J^. In the following, we assume the standard 
monomial basis in each spectral component, and derive 
recursively. We will call this transform QDFT2fc. For k = 1, 
— 1 decomposes over Q as over C, 

C[x]/{x^ - 1) ^ C[x]/{x - 1) ® C[x]/{x + 1). 

Thus, a Fourier transform is QDFTj = DFT2. 
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For an arbitrary size 2*^+^, we use the CRT to get the partial 
decomposition 



C[a;]/(a;2 



l)^C[x]/{x'^ - 1) e C[a;]/(a;2 +1). (78) 



To compute the base change matrix for (78), we determine 
the coordinate vector of each basis elements of the left- 
hand side in (78) w.r.t. the basis on the right-hand side. These 
coordinate vectors are the columns of the base change matrix. 
Namely, for < £ < 2'=, 

x^ = x^ mod x^*' — 1, 



and 



XT = XT mod x"^ + 1, 



= rji-i mod x^'' -1, 



The resulting matrix is 



\.2k 



■ lofc 



DFT2 l2k 



The left side in (78) is decomposed over Q by QDFT2(=+i , the 
right side is decomposed recursively by QDFT2fe ® I2'' (since 
C[a;]/(a;2'° -|- 1) is irreducible). Thus, the recursion for QDFT 
is 

QDFT2.+1 = (QDFT2fc e 12^= ) (DFT2 O l20- (79) 

In summary, QDFT2fc is a Fourier transform for the rational 
finite time model with $ in (77), with the monomial basis 
chosen in each spectral component. Note that QDFT„ is, up 
to a permutation of the columns, equal to the rationalized Haar 
transform (RHT). The RHT is recursively defined as (79), but 
with a permutation multipUed from the right. This implies that 
the RHT could be seen as a Fourier transform for a rational 
signal model with <!>' as linear mapping, which arises from 
$ by permuting the monomial basis. However, we do not 
pursue further this interpretation here since this signal model 
and transform $' are better characterized in the context of 
wavelets and filterbanks. 

Since, the degree of freedom in choosing a Fourier trans- 
form for the rational signal model with $ in (77) is in the 
choice of bases in the spectrum, all its Fourier transforms are 
given by rational matrices of the form 

B-QDFT2., 

where B is rational and a direct sum of invertible matrices of 
increasing block sizes 1,1,2,4,..., 2^~^. 

Diagonalization properties. If /i G Q[a;]/(a;" — 1) is a 
rational filter, i.e., (j){h) is a rational circulant matrix, then 

QDFT„0(/i) QDFT-i=i?, 

where B has the same block structure as above. 



vn. Modeling Space: The C-Transform 

Signal models realized by the infinite and finite z-transforms 
are time models; the signal samples are along an oriented 
time axis, i.e., with intrinsic direction (from past to future). 
Algebraically, this direction is described by the operator q and 
its action o on the time-marks t„ (see (51) and Figure 11). 

In many applications, however, the signal is not sampled 
along time, but along space. Important examples are sequences 
of pixels in images. Space has no intrinsic direction. In the 
absence of further conditions, space is inherently symmetric. 
Thus, a model based on the z-transform is not appropriate. 

We will now develop a space model for signals following the 
exact same steps as in Section V. However, the starting point 
will be different: a shift operation defined to model space. It 
is worth emphasizing this seemingly small but crucial change, 
since the shift in classical signal processing has one and only 
one meaning (51). 

We will see that the realization of this model in terms 
of ordinary multipUcation leads naturally to the Chebyshev 
polynomials and the C-transform that we will introduce. In 
the following section, we will then show that the 16 discrete 
trigonometric transforms (DTT) arise from this model in the 
same way as the DFT arises from the ^-transform. 

A. Building the Signal Model 

Definition of tlie sliift. Analogously to Section V-A, we 
consider discrete signals s e C^, i.e., we consider the vector 

space V = C^. We define now space marks and an appro- 
priate space shift operator q and its operation o on the space 
marks. As mentioned above, q should operate symmetrically. 
We adopt the definition is 



space model: qotn = {tn+i + tn-i)/2 



(80) 



for n G Z. Figure 14 shows a graphical representation of the 
space shift and should be compared to Figure 11. 



qo 



tn-l 



tn+1 



Fig. 14. The space shift qotn- 



As in Section V-A, we proceed by extending the operator 
domain from q to /c-fold shift operators g^. A natural definition 
of the fc-fold space shift is 



qk^tn = {tn+k + tn-fe)/2, 



(81) 



since tn+k and t„-k are those space marks at distance k from 
tk- 

Here we have the first interesting difference with respect to 
the time model derivation, since clearly 7^ g'^. Furthermore, 
(81) implies qk = q-k', hence, it is sufficient to consider only 
shift operators qk with A: > 0. This agrees with our intuition: 
q operates symmetrically on hence there are no negative 
fc-fold space shifts. Thus, the natural representation of a filter 
is X^fe>o hkQk- The following lemma shows that the qk are 
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given by the Chebyshev polynomials of the first kind Tk (see 
Appendix III) in the variable q. 

Lemma 9 The fc-fold space shift operator is given by Qk = 

Tkiq). 

Proof: Induction on k. By definition qq = 1, and qi = q. 
Also by definition, qk+i o tn = {tn+k+i + <n-/c-i)/2 = 

{tn+k+l + tn+k-1 + tn-k+1 + in-fe-l)/2 — {tn+k-1 + 

tn-k+l)/2 = 2qO {tn+k +tn-k) ^2 — {tn+k-1 + tn-k+l)/2 = 

{2qqk — qk-i)otn, for n € Z. From the induction hypothesis, 
Qk = Tk{q), qk-i = 2fe-i(?), and thus, using the recurrence 
of the Chebyshev polynomials introduced below in (149), 
qk+i = Tk+i{q), as desired. ■ 

The Chebyshev polynomials, which just surprisingly 
emerged as the fc-fold space shift operator qk, will play a 
central role in the definition of the space model. For this 
reason, we provide the necessary background on Chebyshev 
polynomials in Appendix III, which we encourage the reader 
to briefly review at this point. 

Linear extension. To construct a linear signal model we 
extend by linearity the operation of q to the entire set M = 
{s = J2nei- ^ntn}, and extend linearly the operator domain 
to A = {h = J2k>o hkTk{q)}- Note that we used the result 
of Lemma 9 in A. 

Realization. Analogous to Section V-A, we determine a 
"realization" of the model introduced in the previous section. 
We set in (80) q = x, o = ■, and determine polynomials C„ 
that replace the space marks t„ in (80), i.e., that satisfy 

X-Cn = {Cn+l+Cn-l)/2. (82) 

Since (82) is equivalent to (149) (in Appendix III), the solution 
is given by a sequence of Chebyshev polynomials. 

We immediately notice differences with respect to the 
corresponding derivation in Section V-A. These differences 
are intrinsic to the space model: 

• Equation (82) is a three-term recurrence for the space 
marks, whereas (52) is a two-term recurrence for the time 
marks. 

• Only the C„, n > 0, are linearly independent; the C„, 
n < 0, are polynomials in x and can thus be expressed 
as linear combinations of {C„ | n > 0}. In other words, 
the realization of the space model introduces a starting 
point in space, given by Co = 1. Fixing C\ determines 
the left boundary condition and the left signal extension. 

• As a consequence, even after normalizing Co = 1, the 
sequence C„ of Chebyshev polynomials is not uniquely 
determined. The degree of freedom is given by the choice 
of Ci as a polynomial of degree 1. 

• Again, we note that in the time model, a /c-fold shift 
operator is given by x'': 

™fc n _ n+k 

in contrast to the space model, where, by Lemma 9, the 
fc-fold shift operator is given by Tk{x), independent of 
C (compare with Lemma 46 iv)): 

Tk - Cn = (Cn+k + Cn-k)/2. (83) 



As a result of this discussion, we obtain the spaces A = 

= Efe>o ^kTk} and = {s = X;„>o SnCn}, i.e., the 
signal model that we obtain later will be only for right-sided 
sequences. 

Table V shows the correspondence between abstract and 
realized concepts. 

To ensure convergence, we would like to require as before 
h e i^{N) and s G However, to prove convergence we 

have first to choose proper boundary conditions, i.e., we have 
to choose the proper Chebyshev polynomials C. We analyze 
the boundary conditions in the next paragraph. This discussion 
has no counterpart in the time model derivation in Section V. 

Left boundary condition and left signal extension. The 
degree of freedom for choosing a Chebyshev sequence C, 
normalized by Co = 1, is given by the choice of Ci, or, 
equivalently, by the choice of C_i, since the entire sequence 
is then obtained by applying the Chebyshev recursion (149) in 
both directions (see Lemma 46, i)). Fixing either Ci or C_i is 
equivalent to choosing a left boundary condition (b.c.) for the 
signal s = (sq, si, • • • )• For example, setting C\ = x impUes 
C_i = x, and thus C_i = Ci, which imposes for the signal 
s the left b.c. s_i = si. Using Table XVI, the corresponding 
sequence is C = T. 

To determine the left b.c. in the general case, we set Co = 1 
and Ci = ax + b, a (to satisfy deg(Ci) = 1). Then, by 
applying (149) backwards, we get 

C_i =2x- (ax + b) = ^—^Ci - —Co. (84) 

a a 

Since C_i is of degree not larger than 1, every polynomial 
C-n, n > 0, obtained by the recursion (149), is of degree not 
larger than n, and thus a linear combination of the polynomials 

C'Oj • • • ) C'ti> 

0<i<n 

which is the left signal extension associated with the sequence 
C. On the other hand, by comparing the degrees of freedom, 
it is obvious that not every signal extension can be obtained 
by choosing a suitable b.c. Thus, 

(C ^=> left b.c.) =^ left signal extension. 

For a generic left b.c, the left signal extension (85) has no 
simple structure; in particular, it is not monomial. Similar to 
Section VI-A, we determine now those left b.c. that yield a 
monomial left signal extension (85). The answer is provided 
in the following lemma. 

Lemma 10 (Monomial left signal extension) Let C = (C„ | 
n e Z) be a sequence of Chebyshev polynomials with Co = 1 
and deg(Ci) = 1. Then the left signal extension associated 
with C is monomial, i.e., every Cfe, fc < 0, is a multiple 
of a C„, n > 0, if and only if C e {T,U,V,W}. The 
corresponding left b.c.'s are given by C_i = Ci,C_i = 
0, C_i = Co, and C_i = — Co, respectively. 

Proof If C e {T, U, V, W}, flien the assertion holds as 
shown in the "symmetry" column of Table XVI. It remains to 
show the converse. We start with the generic left b.c. in (84). 
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TABLE V 

Realization of the abstract space model. 



concept 


abstract 


realized 


shift operator 


1 


Ti (a;) = X 


shift operation 


o 




space mark 


tn 


Cn 


fc-fold shift operator 


Qk = Tk{q) 


Tk{x) 


space shift 


qOt„ = +t„_i) 


X ■ Cn = + C„_l) 


signal 






filter 




T,hkTk{x) 



Because the signal extension associated with C is monomial, 
one of the two summands in (84) has to vanish. 

Case 1: C_i is a multiple of Co, i.e., constant. It follows 
a = 2,Ci= 2x + b, C-i = -b, C_2 = -26a;- 1. Now, either 
C_2 is constant, i.e., b = 0, which implies C = U, ot C_2 is 
a multiple of Ci, which implies b = ±1, or C G {V, W}. 

Case 2: C_i is a multiple of Ci. It follows 6 = 0, (7i = ax, 
C2 = 2a.T2 - 1, C-i = (2 - a)x, a ^ 2, and C-2 = 2(2 - 
a)x^ — 1. Since C-2 has to be a multiple of C2, we get a = 1 
and thus C = T. This completes the proof. ■ 

The four boundary conditions derived in Lemma 10 are the 
discrete versions of the so-called Dirichlet b.c. ("zero value") 
and von-Neumann ("zero slope"), e.g., [80]. In each case, the 
symmetry point is either a "whole" sample point, or a "half" 
sample point, i.e., is located between two sample points. 

After we identified the suitable b.c.'s, we can show that 
filtering is well-defined (i.e, converges). We assume s G 
and h G £^(Z) and consider the example C = T. Using the 
power form of T in (152) in Appendix III, 

h ' s — ^ ^ hjiTji • ^ ^ SjiTji 

n>0 n>0 

n>l 

n>l 

which is the ordinary convolution and exists, since the co- 
ordinate sequences are in £^(Z) and ^^(Z), respectively. The 
resulting Laurent series is again symmetric of the form 

* = (*o + Ey(«" + """))' 

n>l 

and thus the result is t = J2n>o ^nTn and has a coefficient 
sequence t G £^(N). 

Similar computations confirm convergence for C = 
U, V, .W. Note that the existence of the power form and thus 
the monomial signal extension is crucial for this proof. 

Signal model: C-transform. Let (C„ | n G N), C G 
{T, U, V, W}, be a sequence of Chebyshev polynomials. We 
have constructed a signal model for V = which we 

call the C-transform, given by 

$ : S E ^nCn- 
n>0 



The module is given hy M = $(^^(N)) and, independent 
of the polynomials C, the algebra consists of all series A = 
{Yjk>o hkTk] with coefficients h G £^(N). 

We use C as a generic notation, but wiU replace it by either 
T, [/, V, or W, when appropriate, and, accordingly, refer to 
the T-, U-, V-, or VF-transform. 

B. Spectrum and Fourier Transform: DSFT 

Again, we consider the case C = T. First we identify 
the spectrum, i.e., the irreducible modules. Straightforward 
computation shows that each series 

E^{x) = So/2 + E cosna;r„(a;) (86) 

n>l 

is an eigenvector of the shift operator x and thus for aU filters 
in A. Namely, xE^^{x) = cosu)E^j{x), or, more general, 

H{x)E^{x) = H{cosLj)E^{x). (87) 

As in the time case, the E,^ are not in $(£^(Z)), but only in 
$(^°°(Z)); thus, the ^-modules Mu, are not submodules of 
M. 

Another way of obtaining the E^^ is to use again the power 
form (152) of T. Namely, every s = S{x) G A4 (and every 
h = H{x) £ A) can be written as 

S{X) = E ^nTn{x) = So + J2 + 

+ U 



which has the form of a z-transform and shows that E^^ in 
(86) is the sum of two conjugate spectral components (23) 
(in the variable u) in the time case, and, as such, is invariant 
under u and thus under x = {u~^ + u)/2. 

Since u e~^'^ imphes x = {u~^ +u)/2h^ cosu, we get 
the following Fourier transform associated to the T-transform. 

A: M ^ (ea,6[o,.lC)=CM 

s = S{x) I— > S'(cosa;)t^£[o,7r] = w 1-^ ^(cosa;). 

The above derivation also shows the existence of this Fourier 
transform via the existence of the DTFT. The spectrum of s 
can be viewed alternatively as an even function on the circle, 
since ^(cosa;) = S'(cos(— a;)), or as a function on the half 
circle. 
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Further, every spectral component A4ui affords a one- 
dimensional irreducible representation (j)^^ of A. Namely, from 
(87), 

(f>^{H{x)) = H{cosuj). 
The collection, for u G [0,7r], 

{(f)u;{H{x)))^ = {H{cosoj))uj =u>i-^ H{cosu)) 

is the frequency response of the filter h = H{x). 

Similar derivations provide the Fourier transforms for the 
cases C = U,V, W, however the existence for s e i'^{N) is not 
as easily guaranteed. We refer to books on general orthogonal 
series, for example [81], [60]. 

Independent of C, we call this Fourier transform, in analogy 
to the DTFT, the discrete-space Fourier transform (DSFT). 

C. Visualization 

We visualize the space model by a graph using Definition 2. 
However, in contrast to before, there is a difference between 
the graph suggested by the abstract model and its reaUzation. 
Namely, from Figure 14, the graph for the space model should 
look hke Figure 15. As in Figure 10, the edges represent the 
space shift (we drop the common weight factor 1/2 of all 
edges). 

• ■< S- • ■< S- • -« 9- • -6 S- • ■< S- • 

t-2 t-i to ti t2 ts 

Fig. 15. The abstract space model (graphically). 

However, due to the boundary conditions that we needed to 
introduce in the realization, the graph of the realized model 
looks different and is in particular one-sided, i.e., has a left 
border. Corresponding to the four types of b.c.'s that lead to 
a monomial signal extension (Lemma 10), we get the four 
graphs in Figure 16. 
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Fig. 16. Visualization of the space models given by the C-transforms for 
C e {T,U,V,W}. 

For example, in the topmost graph, the extra arrow from 
To to Ti arises from T_i = Ti and the arrow that would 
go from To to T_i. In the second graph, we could have 
omitted the two arrows from U-i to Uq, since their weights 
add up to zero, which is also reflected by C/_i = 0. Including 
U-i gives a better understanding of the signal extension, 
since, following the arrows, each graph not only represents the 
boundary condition, but also the entire left signal extension. 



We will show in Section X that it is possible to choose 
a different realization that has Figure 15 as visualization; 
however, the realization in Section X will raise a different 
difficulty that we will discuss there. 

VIII. Finite C-Transform and DTTs 

In Section VI, we derived the finite z-transform, the finite 
signal model for time, by choosing boundary conditions and 
constructing a polynomial algebra of the form A = C[x]/p{x) 
with basis b = (1, x, . . . , x""^). By requiring a monomial 
signal extension, we obtained p{x) = — a and thus the 
DFT (and its variants) as the associated Fourier transform for 
the regular module M. = A. 

Analogously, we derive now the finite versions of the 
C-transform. By identifying those boundary conditions that 
lead to a monomial signal extension, we obtain 16 specific 
polynomial algebras with bases, whose Fourier transforms are 
precisely the 16 types of discrete trigonometric transforms 
(DTTs) comprising 8 discrete cosine transforms (DCTs) and 
8 discrete sine transforms (DSTs). Using the general results in 
Section III, we then explain DTT domain filtering and derive 
the DTT's diagonalization properties and convolution theo- 
rems. We also explain why the DTTs are almost orthogonal 
and many other of their properties. In particular, we show that 
the associated signal models reveal close relationships between 
certain DTTs, which allow us to divide them into four groups 
of four each. DTTs within the same group can be translated 
into each other at the expense of 0{n) operations. An even 
stronger relationship exists between "dual" DTTs, a notion 
that we will define. 

The derivation follows the same structure as Section VI. 
However, as in the infinite cases (see Sections V and VII), 
there will be important inherent differences between the finite 
time model and the finite space model. 

A. Building the Signal Model 

Shift, linear extension, realization. We consider a finite 
number of space marks to, . . . , tn-i and adopt the space shift 
operator q in Figure 14 and its realization by setting q = x, 
and tk = Ck (a generic sequence of Chebyshev polynomials), 
as derived in Section VII-A. These definitions will need to 
be complemented by appropriate boundary conditions, as we 
discuss next. 

Let s = (so, . . . , s„_i) G C" be a finite sampled signal and 
C a sequence of Chebyshev polynomials. As in Section VI-A, 
a straightforward reaUzation seems to lead to the set of all 
polynomials 

skCk- 

0<k<n 

The set of these sums is the vector space C„[a;] (with basis 
polynomials Ck)', however, this space is not closed under mul- 
tiplication by the shift operator x, and thus it is not a module, 
which means filtering is not well-defined. In pjirticular, the 
problem is with 

X-Cn-l = iCn-2 + Cn)/2^Cn[x], (88) 
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since C„ ^ C„[a;]. Note that, in contrast to (55), multiplying 
the first space mark by T_i = Ti ~ x yields 

:r-Co = (C_i+Ci)/2GC„[x], 

since, by (84), the choice of C already implies a left b.c. So 
the remaining task is to determine the proper right boundary 
conditions. 

Boundary condition and signal extension. To solve the 
problem in (88), we introduce an equation 



TABLE VI 

Period lengths for the 16 monomial signal extensions. 



E 

0<fc<n 



PkCk, or C„-r = 0, (89) 



which is equivalent to the right b.c. 

0<k<n 

As a consequence of (89), using the A:-fold space shift operator 
Tfc (see Lennma 9), we get the series of equations 

Tk-{Cn-r)=Tk-0 = 0, k>0, 

which determine the entire right signal extension. It is obtained 
by applying Lenrnia 46, iv) and reducing Cn+k modulo (C„ — 
r). 

Algebraically, the right boundary condition replaces the 
vector space Cn[x] (with basis b — (Co, . . . , C„_i)) by the 
polynomial algebra M. = C[x]/(C„ — r) (also with basis b), 
viewed as a regular module, i.e., the algebra is A = A4. The 
natural basis of A is given by (To, . . . ,T„_i), regardless of 
the choice of C. 

For a general choice of left b.c. (given by the choice of C) 
and right b.c. (given by the choice of r), the corresponding 
signal extension has a complicated structure. As in Section VI- 
A, we identify those b.c. that lead to a simple, i.e., monomial 
signal extension. Lemma 10 gives already the left b.c. for 
a monomial left signal extension and shows that they are 
obtained by choosing C E {T, U, V, W}. It remains to identify 
the admissible right b.c. We do this in Lemma 11 and show 
that, again, there are 4 choices, which give rise to a total 
number of 16 possibilities — corresponding to the 16 types of 
DTTs as we will see below. 

Lemma 11 (Monomial right signal extension) To assure a 
monomial left signal extension, let C e {T, J7, V, W}. The 
only four right b.c. that yield a monomial signal extension for 

M = C[,x]/27(,x) are C„ = C„_2, C„ = 0, and C„ = ±C„_i, 
which implies p e {C„ - C„_2, Cn, Cn ± C„_i}. 

Proof: Necessarily, the b.c. has the form C„ = aCk, 
7^ fc < n. By multiplying by x on both sides, we obtain 
Cn+i ~ a{Ck+i +Cfc-i) — Cn-i- We determine under which 
conditions the three summands on the right reduce to at most 
one summand. 

Case 1: k ^ n — 1. Then either a = 0, or k = n — 2 and 
a=l. 

Case 2: k = n — 1. Then aCk+i = aCn = c?Cn-\ and 
thus a = ±1. 

It remains to show that these four b.c. yield a monomial 
signal extension, which is done by induction. We omit the 
details. ■ 
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It is interesting to note that the right b.c.'s in Lemma 11 are 

the reflections of the left b.c.'s in Lemma 10. 

We investigate the structure of the signal extension. The 
four left signal extensions, corresponding to setting C = 
T, U, V, W, can be displayed (from top to bottom, respec- 
tively) as follows, using '|' to denote the boundaries: 

. . . \Cn Cn-1 • ■ ■ Cl| 

. . . —Cn —Cn-l\ — Cn-2 . . . — Cq 0| 

...Cn\Cn-l ■■■ Cq\ 

...—Cn\ — Cn-1 ■■■ —Col 



I Co ... Cn- 



left signal extension 



signal scope 



(90) 

We observe that the left signal extension is completely 
determined by the left b.c, up to the occurrence of C„ 
(underlined), which is determined by the right b.c. Similarly, 
we can display the right signal extensions as 

\Cn-2 • • • Co C-i |. . . 

|0 — C„_i . . . — Ci|— Co — C-i ■ ■ ■ 

Go ... ... Co|C-i... 

\ — Cn-l ... — Co|— C-1... 



signal scope 



right signal extension 



(91) 

Again, the right signal extension is completely determined 
by the right b.c, up to the occurrence of C_i (underlined), 
which is determined by the left b.c. In the literature [68], the 
four signal extensions in (90) and (91) are sometimes called, 
from top to bottom: whole point symmetry (WS), whole point 
antisymmetry (WA), half point symmetry (HS), and half point 
antisymmetry (HA). 

Taken together, the left and right b.c. determine both the 
left and right signal extension. 



left and right b.c. 



left and right signal extension. 



By combining the left and right b.c, we get, in all 16 cases, 
an eventually periodic signal extension. The period lengths are 
displayed in Table VI. 

We will show a visualization of the signal extensions, 
similar to Figure 12, after the 16 DTTs have been introduced. 

Signal model: finite C-transform, Let Co,...,C„_i, 
C G {T. U, v. W}, be a sequence of Chebyshev polynomials. 
Further, let p{x) be one of the four choices in Lemma 11. The 
finite C-transform is the signal model {A, A4,^) forV = C" 
with M = A = C[x]/p{x) and the bijective Unear map $ 
defined by 

$ : s ^2 ^kCk e M. 

0<k<n 
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TABLE Vn 

Identities among the four series of Chebyshev polynomials; 

C„ has to be replaced by Tn,U„,V„, W„ TO OBTAIN ROWS 1, 2, 3,4, 
RESPECTIVELY. 





Cn ~ 
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Crt 


Crt ~ On—1 


Cn - 


\- Cn-l 
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(x - l)VF„_i 
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2T„ 






v„ 






V„ 
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■ l)H^n-l 




2(x - l)U„-i 
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This definition requires that the module exphcitly be stated, 
since the 16 candidate modules have only 4 choices of bases. 
As in the infinite case, if C is fixed, we will also refer to the 
finite T-,U-,V-, and T4^-transform. 

It is important to note that in the algebra A = A4 of filters, 
the natural basis is given by the fc-fold space shift operators 
(To, . . . , r„-i), and thus independent of the choice of C, i.e., 
of the basis (or impulses) Ck in the signal space Ai. This is 
different from the finite z-transform associated with the DFT 
(see the end of Section VI-A), where both A4 and A had the 
same basis consisting of polynomials x'^ (representing time 
marks and fc-fold time shift operators, respectively). 

The above definition of the finite C-transform includes 16 
variants, corresponding to the 16 DTTs introduced next. 

B. Spectrum and Fourier Transform: DCTs and DSTs 

In this section, we derive the Fourier transforms for the 16 
types of finite C-transforms defined above and show that they 
are given by the 16 types of DCTs and DSTs. As an aside, in 
doing that, we settle the question why there are 16 DTTs to 
begin with, as the original derivation of the full set of all 16 
[78] does not provide an explanation. 

To compute the spectrum and a Fourier transform using The- 
orem 4, we have to determine the zeros of the 16 polynomials p 
given by C„, C„±C„_i, C„-C„_2, for C e {T, U, V, W}. In 
all cases, the zeros can be expressed in closed form using the 
identities given in Table VII and Table XVI in Appendix III. 
We note that Table VII is a consequence of well-known 
trigonometric identities. 

Instead of computing the spectrum and a Fourier transform 
in detail for all 16 cases, we consider only one representative 
example and then state the result for all 16 DTTs. But first, 
we introduce the DTTs as they are defined in the literature. 

DTT definitions. There are 16 types of discrete trigonomet- 
ric transforms (DTTs): 8 types of discrete cosine transforms 
(DCTs) and 8 types of discrete sine transforms (DSTs). The 
most important to date is the DCT of type 2, which was first 
introduced in [82] and is used in the JPEG image compression 
standard. A complete introduction to all 16 types is in [78]. 
Table VIII gives the definitions of the unsealed version of 
the 16 DTTs. We note that the DTTs of type 1, 4, 5, 8 
are symmetric, and that the DTTs of type 2 and 3, 6 and 
7, respectively, are transposes of each other. We use arable 
instead of roman numbers to denote the type, following [56], 
since it is more convenient when dealing with all 8 types of 



TABLE vin 

; TYPES OF DCTs AND DSTS (UNSCALED) OF SIZE n. THE ENTRY AT 
ROW k AND COLUMN £ IS GIVEN FOR < k,£ < n. 



type 



DCT 



DST 



cos k£- 



cos k(i 
cos(fc - 

cos(fe + |)(£+i)^ 

COS ke^^ 

cosk(e+ ^)-^ 
^' n— 

COs(fc-|- 

cos(fc+i)(£+i)-^ 



sin(fc-t-l)(€ + l)^ 

sm(fc + l)(^ + l)^ 
sm(fe + i)(^ + l)^ 
sm(fc + i)(£+i)f 
sin(fe + l)(^ + l)--^ 

sm{k + l){e+^)^ 

sm{k + ^){£ + l)^ 

sin(fc-hi)(€+i)-4 



DCTs and DSTs. For example, we write DCT-2„ instead of 
DCT™. 

Example: Signal model for DCT, type 2. We work out 
this example in detail. We choose the left b.c. s_i = sq, i.e., 
C_i = Co, which is afforded by the base polynomials C = V 
(see Lemma 10). As right b.c. we choose s„ = Sn-i, i.e., 
Cn = Cn-i> which implies 

P = C„- Cn-l = Vn- Vn-1 = 2{x - l)Un-l 

using row 3, column 3 in Table VII. Thus, we obtain the 
regular module M = C[x]/{x - l)Un-i{x) (the 2 can be 
dropped, as scalar factors do not matter in p). The zeros of 
p{x) — {x — l)Un-i{x) are given by = cosfc7r/n, < 
k < n (from Table XVI). Thus the Fourier transform for A4 
is given by 

A: C[x]/{Cn-Cn-i) - eo<fe<nCN/(a;-afe) 
s = s{x) 1-^ (s(Q:o),...,s(a„_i)). 

(92) 

In matrix form, a Fourier transform for A4 is given by the 
polynomial transform with entries 

To obtain a DCT, we need proper scaling. Namely, using 
Table VIII, we have 

DCT-2„ = diago<fc<„(cos fc7r/(2n)) • [VfXak)], (93) 

and thus, by Theorem 4, DCT-2„ is a Fourier transform for 
the regular module M = C[x]/{x — l)Un-i{x). 

The scaling diagonal in (93) shows the basis chosen on the 
right hand side of (92), namely l/(cosfc7r/(2n)) in the one- 
dimensional module (spectral component) Mk = C[x]/{x — 
cos kir/n), for < fc < n. 

In other words, applying DCT-2„ to a signal s € C" gives 
the spectrum of s with respect to the finite F-transform 



E 

0<fe<n 



SkVk e M, 



where M=A = C[x]/{x - l)C/„_i. 

All DTTs. Similar computations for all 16 cases establishes 
the 16 DTTs as Fourier transforms for the 16 finite C- 
transforms. 
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Theorem 12 (DTTs and polynomial algebras) The 16 DTTs 
are the Fourier transforms for the 16 finite C-transforms. 
The correspondence is given in Table IX as follows. Let 
{A,M,^) be a finite C-transform with M = C[x]/p with 
basis b = (Co, . . . , C„_i). The choice of C (rows of Table IX) 
determines the left b.c. and a scaling function /. The choice 
of left b.c. (four rightmost columns of Table IX) then de- 
termines the polynomial p, given at the intersection of row 
and column. The corresponding DTT is given above p. If 
a = {ao, . . . , ctn-i) are the zeros of p, then 



TABLE X 
Signal extension for the 1 6 DTTs. 



DTT„ = diago<fc<„(/(afe)) • 



(94) 



i.e., DTT„ is a scaled polynomial transform and thus a Fourier 
transform for the associated signal model (see Theorem 4). 
Equation (94) implies that the chosen basis in the spectral 
component 0o<fe<„ C[a;]/(a; - ak) is l//(afe), < < n. 

The DCT, type 3, was recognized as polynomial transform 
in [541. The DCTs and DSTs of types 1-4 where recognized 
as (scaled) polynomial transform in [83]. In neither case any 
connection to signal processing was established. 

Polynomial DTTs. Theorem 12 shows that each DTT is a 
Fourier transform for a suitable module C[x] /p, but, in general, 
not the polynomial transform. Thus, we associate to each DTT 
its polynomial transform V^^a obtained by omitting the scaling 
factors in (94). 

Definition 13 (Polynomial DTTs) Let DTT„ be given. We 
call the unique polynomial transform Vb,a associated with 
DTT„ by (94) the "polynomial DTT" and denote it by DTT„. 
Thus, (94) can be rewritten as 



DTT„ 



diag. 



0<fe<n 



ifiak)) ■ DTT„ 



We have DTT = DTT if and only if DTT 
in the first row of Table IX, i.e., if DTT is 
DCT-1, DCT-3, DCT-5, DCT-7. 



appears 
one of 



The polynomial DTTs will play an important role in the 
derivation of fast DTT algorithms [14]. In several cases it will 
be natural to derive a fast algorithm for DTT and apply a final 
scaling to obtain a fast algorithm for the corresponding DTT. 
As a consequence, we will show that the polynomial DTTs are 
a suitable choice of scaled DTTs in applications, where the 
DTT is followed by scaling and can thus be replaced by any 
transform D ■ DTT to reduce the number of multiphcations. 

Signal extension. We display graphically the signal exten- 
sion for the 16 DTTs in Table X. This is similar to Figure 12 
for the DFT variants. The dotted lines (including the adjacent 
hollow bullets) signify a scaling by —1. A "0" signifies that 
the signal model assumes a signal value equal to zero. In each 
case we display four times the signal scope, which may be a 
single period or comprise two periods, depending on the DTT. 
For some of the bullets, the labels at the (virtual) x-axis are 
given. 

Remarks and observations. We make the following re- 
marks. 

• For each DTT, we have three relevant versions. First, 
the polynomial version DTT, which is the unique poly- 
nomial transform for its associated signal model (see 
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Definition 13 above). Second, the unsealed or natural 
version, which has pure cosines (or sines) as entries 
(see Table Vlll). Third, the orthogonal version, which 
arises from the other two by suitable scaling of rows 
and columns, i.e., by slightly adjusting the signal model 
(explained below in Section VIIl-F). 

• The 16 DTTs can be divided into four groups of four 
each with respect to the polynomial p in the associated 
module C[a;]/p (see Table IX). For example, the T-group 
comprises all DTTs of type 3 and 4, which have the 
same module M. = C[x]/T„. The modules within the 
other groups differ slightly, e.g., in the [/-group that 
comprises the DTTs on the main diagonal in Table IX. 
The difference between the DTTs within the same group 
is the choice of basis, which is one of T, U, V, W . As 
a consequence, these transforms can be converted into 
each other using a sparse base change (explained in 
Section VIII-H), and several convolution theorems can 
be derived within a group (explained in Section Vlll-E). 

• A closer relationship exists between the DTTs that are 
in the same group but, in addition, on mirror positions 
w.r.t. the diagonal. In other words, these DTTs have 
mirrored b.c. We call such a pair dual to each other 
and show a uniform relationship between dual DTTs (see 
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TABLE IX 

Overview of the 16 DTTs and the associated modules C[x]/p{x) with a basis of Chebyshev polynomials that admit a monomial 

SIGNAL EXTENSION. The LEFT B.C. (ROWS) DETERMINES A SCALING FUNCTION / (cos 6* = x) AND THE CHEBYSHEV POLYNOMIALS C G {T, U,V,W}. 
The RIGHT B.C. (COLUMNS) THEN DETERMINES THE DTT AND p{x) (GIVEN BELOW THE DTT). 
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Section VIII-G). 

C. Visualization 

Since the right b.c. for the 16 DTTs are precisely the mir- 
rored left b.c, we obtain the visualizations (see Definition 2) 
of their associated signal models readily from Figure 16. We 
show four important cases in Figure 17, from top to bottom 
the DCTs of type l-A. 

• ■< • • ■< ^ • • 

II /-T^ /-T^ rr\ rri '"'T"' 

J-1 J-2 -l-n-S J-n-2 -Ln-1 

r • -e s- • -iS s- • • s- • -« 9- • J 

Vo Vi V2 Vn-3 Vn-2 K-1 

■< s- • • -s s- • -* s- • 

Jo Jl J2 Jn-3 Jn-2 -ln-1 
• -< s- • ^ *- • • -< s- • -i s- • 

Vo Vl V2 Vn-3 Vn-2 

Fig. 17. Visualizations of the space models given by the C-transforms and 
associated with the DCTs of type 1^ (from top to bottom) and size n. 

Since the graph represents the operation of the space shift 
operator x = Ti, the adjacency matrix of the graphs in 
Figure 17 is in each case given by the shift matrix ^{x) {(j) is 
the representation afforded by the respective signal model). 

D. Diagonalization Properties 

Using the algebraic framework, the diagonalization proper- 
ties of the 16 DTTs can be easily derived from Theorem 5 and 
can be stated in a unified way for all 16 DTTs. Let DTT„ be 
given and let ^ = = C{x\/p{x) be the associated regular 
module with basis h and let a = (ao, . . . , be the vector 

of zeros of p. Denote by the corresponding representation 
of A. Then, for a filter h e A, 



,^{h{au)). (95) 



diago<fe<r 

Conversely, the ^(/i) are all the matrices diagonaUzed by 
DTT. 

We first investigate the special case of the space shift 
operator h = Ti = x. Since A, as any polynomial algebra, is 
generated by the shift operator x, the diagonalization of the 



TABLE XI 

The values /3i , /32 , /^s , f^4 from (97) for the 4 respective choices 

OF LEFT B.C. AND RIGHT B.C. 
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shift matrix (f)(x) impUes the diagonalization of all matrices 
(t>{h), he A. Using 

x-Ct = {Ct+i + Ct-i)/2, (96) 

we obtain as the general structure of (/)(a;) for all 16 cases 



cl>{x) 







/33 

Pi 



(97) 



where the numbers /3i , P2 and (3^ , P^ are determined by the left 
and right b.c.'s, respectively, i.e., by (96) for = 0, n— 1. For 
example, for DTT = DCT-2, we have C-i = Co (i.e., C = 
V) and thus x ■ Co = {Co + (7i)/2 or /3i = /Ja = 1. Further, 
Cn = Cn-1, and thus x ■ Cn-i = {Cn-2 + C„-i)/2, or 
/?3 = /34 = 1. The resulting shift matrix (^(a;) is the adjacency 
matrix of the second graph in Figure 17. 

Table XI lists the values of the 1 < z < 4, in all 16 
cases. The property 

DTT„ ■(l){x) ■ DTT-i = diag(ao, • • • , 

is, in a strict mathematical sense, the analogue of the DFT 
diagonalizing the cyclic shift. 

Next, we investigate the special case of a fc-fold space shift 
operator h = Tk. We have, by Lemma 46 iv). 



Tk- Ce = (C^+fc -I- Ce-k)/2. 



(98) 



Further, in the cases where £ + k or i — kis outside the range 
0, . . . , n — 1, the monomial signal extension characteristic to 
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the 16 DTTs assures that TkCe is again a sum of only two 
basis polynomials in (Co . . . C„_i). Thus, the matrix (j){Tk) 
has at most two entries per row and column, and further has, in 
all 16 cases, a "rhombus-like" shape. We illustrate this again 
for DTT = DCT-2. Using (98) and row 3 of (90) and row 3 
of (91), we get 



1 

where the dots signify I's; the rest of the matrix entries are 
zero. In this rhombus-shaped matrix, the upper left and lower 
right side of the rhombus are due to the symmetric boundary 
conditions. 

In the general case, h = J^'^kTk, we get by linearity 
= J2 o-k4'{Tk)- Thus (pQi) is a structured matrix obtained 
by a generic linear combination of the matrices <j){Tk). Taken 
together we obtain the diagonalization properties stated in [84] 
(which considers only the 8 DCTs) and, in addition, give 
insight into the structure of the matrices and explicitly give the 
obtained diagonal matrix in (95). This makes it possible, for 
example, to determine whether a given matrix 4>{h) is positive 
definite. For example, [56] uses ft, = 2 — 2a; to illustrate the 
different types of DCTs. Using (95), the eigenvalues of 4>{h) 
in all 16 cases are of the form 2 — 2a > since the zeros of 
p in Theorem 12 are all cosines. Thus (/)(2 — 2x) is positive 
semi-definite. In general, we have the following result. 

Lemma 14 Let M = C[x]/p{x) with basis b and (p the 
afforded representation of A = Ai. Let h G A. Then (f>{h) 
is positive definite (semi-definite), if h{ak) > (> 0) for all 
zeros ak of p. 

Finally, we note again that the structure of the matrices 

(f>{h), h E A, reflects the basis (To, . . . , r„-i) of A, chosen 
independently of M. The Tf. are the fc-fold space shift 
operators. 

E. Convolution Theorems 

With the underlying signal models for the 16 DTTs identi- 
fied, we obtain a natural, unified description for DTT domain 
filtering. Let DTT„ be given with associated regular module 
Ai =^ A = C[x]/p{x) with basis b (Theorem 12). Then 
filtering with respect to the signal model given by the A- 
module Ai is, as usual, the multiplication of a polynomial 
s £ A4 (the signal) by a polynomial h G A (the filter) modulo 
p. In coordinate form, we have, also as usual, 

h ■ s mod p ^ (/)(/?,) • s, 

where is the representation of A afforded by M with basis 
b. We determined the structure of 4>{h) in Section VIII-D 
w.r.t. the (natural) T-basis in the algebra A of filters. 



A convolution theorem for DTT domain filtering is now 
obtained as a special case of Theorem 6. We illustrate with two 
examples. In the first example, we choose p = T„ and M. = 
C[x]/Tn with F-basis^^ and associated Fourier transform = 
DCT-4„. As above, let s be the coordinate vector of the signal 
s. Further, let /i e ^ be a filter. The Fourier transform J^' in 
Theorem 6 is w.r.t. the basis in A, for which the natural choice, 
as we learned, is always the T-basis. Thus J^' = DCT-3„, 
which has no scaling diagonal. Let h be the coordinate vector 
of h, then 

^{h) ■ s = DCT-4-1 • (DCT-3„ • h DCT-4„ • s). (99) 

We can choose a basis in A different from the natural T- 
basis, to obtain variants, where DCT-3 in (99) is replaced by 
the DTTs in the T-group, and (99) is modified to account for 
their corresponding scaling diagonals D (Theorem 6). 

In the second example, we choose p ~ {x — l)?7„_i and 
A4 = C[x]/(x — l)Un-i with F-basis and associated Fourier 
transform DCT-2„. Proceeding as above leads to the problem 
that for A = A4 with T-basis there is no associated DTT. 
Namely, the DTT in the [/-group with T-basis is the DCT-1, 
which has the associated module A4 = C[x]/{x'^ — l)C/„_2, 
which differs from the Ai above by a linear factor in p. 

To obtain a Fourier transform !F' for C[a;]/(a; — l)C/„-i 
with T-basis, we thus split off the extra linear factor and start 
with the module for DCT-l„+i to obtain comparable sizes. 
Namely, using the CRT, 



[x]/(x^ -l)Un- 



[x]/(x-l)Un-l®C[x]/(x+l). 



We choose the T-basis in the smaller modules. The corre- 
sponding base change matrix Bn+i has the form 



1 



1 * 



where * denotes entries whose exact form we do not need. 
We get 

dct-i„+i = (:f'©Ii)b„+i. 

The special form of B„+i allows us to use the DCT-l„+i in 
a convolution theorem as 

(p{h)-s = DCT-2,71 • (J^' •h0DCT-2„ -s) 

= DCT-2,;i • ((DCT-l„+i • h')" DCT-2„ • s), 

where the h' arises from h by appending (padding) a zero 
value, and (•)" signifies omitting the last value. 

The DTTs can also be used to compute the "ordinary" linear 
convolution, i.e., time domain filtering, if the signal or the 
filter have a symmetry property compatible with a DTT. This 
is the subject of [68]. That paper also observes that the 16 
DTTs divide into four groups of four each of "compatible" 
DTTs, i.e., DTTs that can occur in one convolution theorem. 
These groups are the same as we introduce in Section VIII-H, 

^^The reader may have noticed that the symbol V is used to represent either 
the vector space in the signal model or one of the Chebyshev polynomials. 
The context should make it clear which meaning is attached to V. 
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TABLE Xn 

Definition of the orthogonal versions of the DCTs and DSTs; 

ai^ i IS the entry at row k and column I of the respective 

UNSCALED DTT AS GIVEN IN TABLE VIII. THE ROW/COLUMN SCALING 
factors ARE GIVEN BY: Cj = 1 /a/2 FOR 1 = AND = 1 ELSE; 
di = l/-v/2F0Ri = n — 1 AND = 1 ELSE. 



type 



DCT 



DST 




CkCidkdi ■ ak I 



_i/2 ' '^k'^t ■ ak,l 
-p2 ■ <^kd.i ■ ak^i 
dkce ■ ak,i 



+1/2 




0.k,l 

dkde ■ ak,i 



namely those with (essentially) the same associated module, 
which explains this observation. Further, the necessity of zero 
padding and last value omission in various cases in [68] has 
the same origin as in our above DCT-2 convolution theorem; 
it occurs when the modules for the two DTTs used in the 
convolution (and thus being necessarily in the same group) do 
not have the exact same associated module. 

F. Orthogonal DTTs 

It is well-known that the DTTs, as defined in Table VIII, are 
"almost orthogonal," which means that after a suitable scaling 
of rows and columns they become orthogonal. Table XII gives 
these orthogonal versions of the DTTs, which arise from the 
unsealed version by scaling the first or last row or column by 
a factor of l/\/2, and by multiplying the entire matrix by a 
suitable scalar factor. We call them orthogonal DTTs. 

A natural question to ask is how these scaling factors are 
obtained. In Theorem 12 we established that the 16 DTTs 
are Fourier transforms for regular modules M. =C[x\/p with 
a basis of Chebyshev polynomials. However, if M. is given, 
the natural choice for a Fourier transform is the polynomial 
transform DTT introduced in Definition 13. The scahng 
functions / to obtain the DTTs from the DTTs were found 
by observation, i.e, by comparing with their definition. 

In this section we derive the orthogonal DTTs from their 
polynomial counterparts, i.e., we compute diagonal matrices 
Di,D2, such that 

Di - DTT -Da 

is orthogonal. In the case when the matrix D2 in this scaling 
is necessary, it is immediately clear (from Theorem 4) that 
the underlying signal model has to be modified to admit the 
orthogonal DTT as Fourier transform. We derive these signal 
models and show that they are symmetric in a sense that will 
be defined. 

Derivation of scaling factors. The root of the "almost or- 
thogonality" of the DTTs is the following Christojfel-Darboux 
formula for orthogonal polynomials [85]. 



Theorem 15 (Christojfel-Darboux formula) Let [Pu | A; > 0) 
be a sequence of orthogonal polynomials over / c R with 
respect to some weight function uj{x), i.e., {5 denotes the 
Kronecker delta function) 



Pk{x)Pc{x)u){x)dx = Hkhi, A*n > 0. 



Further, denote with (3k the leading coefficient of P^. Then 

E i^k'Pkmiy) 

0<k<n 

Pn-l{y)Pn{x) - Pn{y)Pn-l{x) 

X-y , ^^y, (joo) 

^c„(P„_i(x)P;(x) - P„(.t)P;_i(,t)), X = y, 

where the constant c„ is given by c„ = /3„_i/?~^/u~li, and 
Pi denotes the derivative of Pfc. 

As a consequence of the Christoffel-Darboux formula, we 
get the following construction method for orthogonal versions 
of polynomial transforms. This theorem, as the Christoffel- 
Darboux formula, is more general than needed here. We will 
need the full generality for the generic next-neighbor model in 
Section XII. We also note that the following theorem restates, 
in algebraic terms, the construction method for unitary trans- 
forms from [86], where it is called Gauss-Jacobi procedure. 

Theorem 16 Let (P^ | A: > 0) as in Theorem 15, and let 
M = C[x]/Pn with basis b = (Pq, . . . , P„-i). Since P,i is 
an orthogonal polynomial, it is separable (see [87, p. 28]); we 
denote its list of zeros by a = (ao, . . . , cen-i)- Further, we 
define the two diagonal matrices 



D = c-idiago<fe<„((P„-i(afc)P^(afe))-i) 
E = diago<fc<„(/ifc^), 

where Cn,^k are defined as in Theorem 15. Then, 
VD ■ Vb,a ■ \fE 

is orthogonal. 



(101) 



(102) 



Proof: First, we note that D is well-defined: P„_i(Q!fe) ^ 

0, since P„_i and P„ have disjoint sets of zeros ([87, p. 28]); 
P^{ak) 7^ 0, since P„ is separable. Now, we substitute the 
zeros ai, aj for a;, y in (100) to get 

which implies the desired result. ■ 
This general property explains the form of the orthogonal 
versions of the DTTs in all cases in which the associated 
module has the form €.[x]/Cn with basis (Co, . . . , C„-i), 

1. e., for all DTTs in the second column in Table IX, i.e., for 
DCT-3, DST-1, DCT-8, and DST-6. For the other 12 DTTs, 
we need to derive variants of (100). We have deferred this 
rather technical derivation to Appendix IV. 

The above derivation of the orthogonal DTTs gives little 
intuition into this, at first glance, surprising property. However, 
the property is easy to understand and easier to derive by 
looking at the diagonalization properties of the DTTs, as 
was pointed out in [56]. We explain this next, and give 
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further insight by constructing the signal models, for which 
the orthogonal DTTs are Fourier transforms. 

The orthogonal transform in (102) is scaled from both 
sides by a diagonal matrix. Scaling on the left side, as we 
have shown, is equivalent to choosing a different basis in 
the spectrum, i.e., in the decomposed module (Theorem 4). 
Scaling on the right changes (by scaling) the basis in the 
module and thus the signal model. We explain this using the 
DCT-3 as an example, and, at the same time, motivate the 
origin of the diagonal matrices. 

Signal model for the orthogonal DCT, type 3. The 
DCT-3„ is a Fourier transform for the signal model 

$ : s ^ SkTk eM= C[x]/Tn, A = M. 

0<k<n 

If (j) is the associated representation of A, then DCT-3„ 
diagonalizes any matrix (t>{h), h £ A. In particular, 

1 
2 1 
10- 

1 • 1 
• 1 
1 



^/2 



<j>{x) = 



(103) 



which is a special case of (97). 

We observe that the matrix in (103) is "almost" 

synnmetric. In fact, symmetry can be readily established by 
conjugating <p{x) with the diagonal matrix 

E = diag(\/2,1,...,1), 

i.e., E(f){x)E~^ is symmetric. This corresponds to the follow- 
ing base change in the underlying module: 

6 = (To, ... , r„_i) ^ &' = (^To, Ti, . . . ,r„_i). 

If (p' is the representation afforded by b', then 

V2 



(p'ix) = E<p{x)E- 



V2 




1 

1 

1 



The base change with E also changed the underlying signal 
model, which is now given by 

s^so^To+ SkTkGM = A = C[x]/Tr,. 



The change can also be seen in its visuaUzation in Figure 18 
(as usual, the global factor 1/2 was omitted), which should 
be compared to the visuaUzation of the original model $ in 
the third graph in Figure 17. The new model has stiU the 
same left b.c. and signal extension, but, in order to make it 
symmetric, the space shift was locally redefined, which means, 
at these marks, the model is now variant. Namely, for Tq = 
l/V2To,TLi=Ti,T^eb' as 

xTi = i(^/2T^ + T^). 



Tn 



Ti 



. • ... 

T2 



T, 



n— 3 



Tn-l 



Fig. 18. Visualization of the symmetric signal model associated with the 
orthogonal version of the DCT, type 3. 



To compute a Fourier transform for we use the following 
commutative diagram. In the top row, the modules have the 
bases b and 6', respectively. 



C[x\/Tn ■ 



E 



■ C[x]/Tn 



Vb,a = DCT-3„ 



Vb',c 



■^C[x]/{x-ak) 



^C[x]/{x-ak) 

In particular, 

DCT-3„ = Vb,a = Vb',aE. 

At this point, we remind the reader that every synmietric 
matrix A can be diagonalized by an orthogonal matrix M 
(e.g., [88]): MAM~^ is diagonal. Further, if A has pairwise 
distinct eigenvalues, then all matrices that diagonaUze A have 
the form DM, where D is any invertible diagonal matrix. 
In our particular case, Vb',a diagonalizes a symmetric matrix 
with pairwise distinct eigenvalues, thus, 

DBCT-3r,E-^ 



M = DVb'. 



(104) 



is orthogonal for a suitable diagonal matrix D that normalizes 

the row vectors of Vb',a to have length 1. The matrix M is 
a Fourier transform, but not a polynomial transform, for the 
signal model 

Signal models for all orthogonal DTTs. In the general 
case of a DTT, i.e., a (/>(a;) in (97), conjugation, E(l){x)E~^ , 
with the matrix 

£ = diag(V^,l,...,l,v/;5^) 

makes (t){x) symmetric and we get the following theorem. 

Theorem 17 (Signal models for orthogonal DTTs) Let 
DTT„ be any of the 16 DTTs with signal model {A,M, $) 
where M = A = <C[x\/p{x) with p one of the polynomials 
in Lemma 11, and 



SK^ ^ SfeCfc. 



0<fe<n 



Let 4'{x) be the associated shift matrix given by (97). Further, 

let /32, A3 be the values in (97) for (t>{x). If DTT'„ is the or- 
thogonal version of DTT„, then DTT^ is a Fourier transform 
for the signal model {A,M, with 

: s 1-^ So— 



fW2 



SkCk + Sn-l—l=Cn-l- 
l<fe<n-l '^'^^ 



We call the signal models symmetric in the following sense. 
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Definition 18 (Symmetric signal model) Let {A,A4,^) be a 
finite shift-invariant signal model with A4 = C[x]/p{x), and 
let (I) be the afforded representation of A. We call this model 
symmetric, if is symmetric for all filters h G A. 



Note that if ^ = A^, then the model is symmetric if and 
only if (j){x) is symmetric. (If A is smaller than M, then 
X ^ A; we will have an example of this situation later.) 

The above discussion explains how to easily obtain, for a 
given DTT, the diagonal matrix E that scales from the right. 
The matrix D is then obtained from the norms of the DTT's 
row vectors using basic trigonometric identities. 

The adjustment of the signal model is necessary, because 
the matrices (j){x) in (97) are not all symmetric due to their 
boundary conditions. Closer inspection shows that among the 
four types of boundary conditions only one, namely s_i = si 
on the left, and s„ = s„_2 on the right, causes this problem. 
Thus, the only DTTs that need an adjustment of the signal 
model are those in the first row or column in Table IX. All 
other DTTs need to be scaled only from the left as can be 
confirmed from Table XII. 



G. Duality 

We observed before that the right b.c.'s for the DTTs are 
precisely the mirrored versions of the left b.c.'s, a fact that 
meets our intuition since the DTTs are based on symmetric 
space models. However, the construction of the module C[x]/p 
for a given DTT (see Theorem 12) deals differently with the 
left b.c. (which determines the choice of the base sequence 
C) and the right b.c. (which determines p); thus, we obtain 
different DTTs for a given pair of b.c.'s and for its mirrored 
counterpart. The associated pair of DTTs occurs in positions 
in Table IX that are mirrored at the main diagonal. We call 
such a pair dual and show that dual DTTs have a very close 
relationship. 



Definition 19 (Duality) Let DTT and DTT' be at mirrored 
positions in Table IX, i.e., at positions («, j), (.?,«), 1 < i.j < 
4, respectively. We call DTT and DTT' dual to each other. 
The DTTs on the main-diagonal are called self-dual. Dual 
DTTs have the same associated regular module C[x]/p. 



To derive the relationship between dual DTTs, we use 
DCT-3„ and DST-3„ as an example. The module A4 = 
C[x]/p associated with DCT-3„ carries the left b.c. ,s_i = si, 
determined by the T-basis b = (Tq, . . . , T„_i), and the right 
b.c. Sn = 0, determined by the equation p = T„ = 0. The 
zeros of T„ are given by a = (cos(fc + l/2)7r/n | < fc < n). 
Now, we consider the same module M, but choose a different 
basis, namely b' = {Un-i,- ■ ■ , Uq). Our goal is to construct 
the following diagram; C[x]/p with basis b' is the top left 



module. 



C[x]/p- 



■C[x]/p 



DCT-3, 



®C[x]/{x~ak) 



Dn 



DST-3^ 



■®C[x]/ix-ak) 



We know how to decompose M with b', namely by first 
reversing the order of the basis with a permutation J„ (the 
identity matrix with the columns in reversed order) to obtain 
the top right module, which in turn is decomposed by DST-3„, 
where the decomposed module has the basis (l//(afc) | < 
k < n), where / = sin 6', cos9 = x (see Theorem 12). 

On the other hand, M. with b' affords the same representa- 
tion as M. with b. To see this, we Ust b and b' together with 
their b.c.'s; the vertical lines indicate the boundaries of the 
signal scope: 

r_i = Ti I To ... Tn-1 I T„ = 

Un = Un-2 I Un-l ... C/q | C/-1 = 

Note that r„ = 0, expressed in [/-polynomials, becomes Un = 
Un-2, which is the right b.c. of DST-3 and the mirror image 
of the left b.c. T_i = Ti of DCT-3. Observe, e.g., that in M 
xUn-i = iUn + Un-2)/2 = Un-2 and xTq = (T_i+Ti)/2 = 
Ti, i.e., x has the same effect on b as on b'. 

Since b' affords the same representation as b, M with b' is 
also decomposed by the DCT-3 (left column in the diagram); 
it remains to determine the basis in the decomposed module. 
Note that DCT-3 is a polynomial transform for A4 and b, but 
not for M and b'. 

In fact, using 

Un-lTi = [Un-l-i + Un-l+i)/2 = Un-l-i mod T„, 

which is due to the signal extension of DST-3„, we get 

b' = Un-lb. 

This shows that the basis in the decomposed module (bottom 
left) is given by the numbers 

C/„_iK) = sin(fc + l/2)7r//K) = (-1)V./K), 



where / is the scahng function of DST-3 (see above). This 
completes our diagram with £>„ = diago<fe<„((— I)*'). As an 
equation, we get 



diagfc^olC-l)") • DCT-3„ = DST-3„ . J„ . 



(105) 



Analogous computations verify the same identity for all pairs 
of dual DTTs. Interestingly, the diagonal £>„ is in all cases 
the same. Also, we note that = D"^ = In. 

Theorem 20 (Duality Relationship) Let DTT„ and DTT'„ be 
a pair of dual DTTs. Then 

diago<fc<„((-l)') • DTT„ = DTT^ . J„, 
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with corresponding diagram 



matrix S'^ for 6 — > 6' is given by 



C[x]/p- 



DTT„ 



■ C[x]/p 
DTT' 



diag((-l)h 

C[x]/{x - ak) i C[x]/{x - au) 

As an important consequence of Theorem 20, dual DTTs have 
the same arithmetic complexity. 

In the literature, the special case of the duahty (as defined 
by us) between DCT-3 and DST-3 was the subject of [89]. 



H. Groups of DTTs and Relationships 

In Section VIII-G we introduced the concept of dual DTTs, 
which necessarily have the same associated module C-[x\/p. 
However, in Table IX, we also have DTTs with the same 
module, which are not dual. An example is given by the DCTs 
of type 3 and 4 with module C [x] /Tn.ln fact, closer inspection 
of Table IX shows that, barring linear factors, each of the four 
types of Chebyshev polynomials occurs exactly four times as 
p. For example, p = Tn occurs for all four DCTs and DSTs 
of type 3 and 4. Thus, we have a natural classification of the 
16 DTTs into four groups of four each. We call these groups, 
depending on p, T-group, [/-group, y-group, and M^-group. 
In particular, dual DTTs are in the same group. 

Further inspection shows that, in each group, all possible left 
and right b.c.'s are present. Thus the DTTs in one group have 
(almost) the same module, but with different bases. Thus, we 
can translate DTTs in the same group into each other using a 
base change. Further, because of Table Vll, the resulting base 
change matrices are sparse, i.e., require 0{n) operations for 
multiplication. 

Before we give two instructive examples, we remind the 
reader that decomposing a regular module C[a;]/p by a 
polynomial transform implies that in each one-dimensional 
irreducible summand C[a;]/(a; — ak) the basis (a;'^) = (1) 
is chosen; other choices of base elements 7^ 1 lead to a 
scaled polynomial transform (see Theorem 4). The DTTs are 
scaled polynomial transforms; their polynomial counterparts 
are de noted by DTT (see Definition 13). We have DTT = 
DTT if and only if DTT is a DCT of odd type. 

Example: DCT, type 3 and 4. First, we consider DCT-3„ 
and DCT-4„, which are both in the T-group, i.e., the associ- 
ated module is = C[a;]/T„. The difference is in the choice 
of bases. 



DCT-3„ : 
DCT-4„ : 



<C[x\/Tn, 



h -- 
b' 



(To, . . . , r„_i), 

= (l/o,...,K-i). 



Using 



Tc = {Ve + Vt-i)/2 



(106) 



from Table VII and y_i = Vq, the corresponding base change 



2 1 
1 1 



1 1 
1 



(107) 



We denote the zeros of Tn by au = cos(fc + l/2)iT/n. As a 
consequence of the above, we get the commutative diagram 



C[a;; 



/Tn 



DCT-3„ 



■C[x 



®C[x]/{x-ak) 



In 



/Tn 



(108) 



DCT-4„ 



■®C[x\/{x-ak) 



which implies the equation 



DCT-3„ = DCT-4„ • 

Note that we have I„ in the bottom row of (108) since both 
DCT-3 and DCT-4 are polynomial transforms and thus use 
the same basis (1, . . . , 1) in the decomposed module. 
Introducing the scahng diagonal 

Dn = diago<fc<„(cos(2A; + l)7r/(4n)) 

of the DCT-4 (see Table IX), we get 



C[a;; 



/Tn 



■C[x 



/Tn 



DCT-4„ 



0C[x]/(x-afe) 
or, as an equation, 

Dn ■ DCT-3„ = DCT-4„ • S'^. 



^<C[x\/{x-ak) 



(109) 



We use (109) to also show how formal manipulation derives 
new relationships from known ones. Since it is known that the 
DCT-4 is more expensive to compute than the DCT-3, we 
multiply (109) by {S'„)~'^ to get 



Dn ■ DCT-3„ • = DCT-4„. 



(110) 



We can multiply by (S'^j)"^ using n — 1 recursive additions, 
but this produces a critical path of length n — 1. To solve the 
problem, we invert both sides of (1 10) using DCT-3 ~^ = 2/n- 
diag(2, 1, . . . , 1) • DCT-2„ and DCT-4- ^ = 2/n • DCT-4„ 
(follows from their orthogonal versions in Table XII and 
Table VIII). It turns out that some factors cancel each other, 
and we get 



Sn ■ DCT-2„ • Id-^ = DCT-4„ 



(111) 
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with 



1 1 

Oil 



1 1 
1 



(112) 



Transposing (111) yields 
1 



-D-^-DCT-3„-5j = DCT-4„, 



(113) 



where we used that DCT-2^ = DCT-3„ and that DCT-4„ 
is symmetric (Section VIII-B); (113) is a clear improvement 
over (110) obtained by inversion-transposition. 

Example: DCT, type 1 and 2. As a second example, we 
derive a relationship between DCT-l„+i and DCT-2„, which, 
without the module framework, is less obvious to derive. Note 
the difference in size (n + 1 versus n) to obtain comparable 
modules, which are 

DCT-l„+i: C[x]/ix^-l)Un-i, 6 = (Tq, . . . , T„), 
DCT-2„ : C[x]/{x - l)[/„_i, b' = (Fo, . . . , K-i)- 

To translate a DCT-l„+i into a DCT-2,,, we have to partially 
decompose the module C[x]/{x^ — l)t/„_i using the Chinese 
remainder theorem, 

C[x]/{X^ - l)Un-l C[x]/{X - l)Un-l ® C[x]/(x + 1). 

As bases in these three modules we choose, from left to right, 
b, b' , and (1). We compute the corresponding base change 
matrix by using (106) and Ti mod (x + l) = Tt{-1) = (-1)^ 
from Lemma 47, iii) in Appendix III. The result is 



B. 



n+l 



1 
1 2 



2 -2 



The last colunm uses the b.c. Vn = Vn-i- Consequently, we 
get 

DCT-U+i = (DCT-2„ © Ii) • B„+i. 

Using the scaling diagonal Dn — diago<fe<„(cos fc7r/(2n)) of 
the DCT-2, this can be translated into 



[Dn e Ii) • DCT-l„+i = (DCT-2„ Ii) • B^+i 
or, as a commutative diagram, 

Bn+l 



(114) 



C[x\/{x^ - 



n-l 



<C[x\l{x - l)C/„_i 

e c[x]/{x + 1) 



Base change theorem. Using base changes between DTTs 
of the same group, combined with transposition, and inversion, 
we get the following theorem. Note that the sparsity of the base 
change matrix is guaranteed by Table VII. 

Theorem 21 (Base Change Theorem) By a base change in the 
associated module, all DTTs of type 1-4, and all DTTs of type 
5-8 can be translated into each other using 0{n) arithmetic 
operations. 

/. Real and Rational Signal Model 

In Section VI-G, we showed the effect of reducing the base 
field from C to M in the finite time model. Doing the same in 
the finite space models, in contrast, does not incur any change, 
since all considered polynomials factorize completely over M. 
Thus, the real finite space models will share all the properties 
from their complex counterparts. 

The situation is different, if we further restrict the base field 
to Q. The question becomes how the polynomials in Table IX 
factor over Q. This question is answered in [90] and can be 
used to derive rational versions of the DTTs. An application 
could be the derivation of algorithms, but we did not pursue 
this direction. 

IX. Finite Skew C-Transform and Skew DTTs 

In this section we introduce a new class of transforms that 
is closely related to the DTTs. We call these transforms skew 
DTTs. More specifically, the skew DTTs correspond to and 
generalize the DTTs in the T-group, i.e., those with associated 
module C[x]/T„, which are the DCTs and DSTs of type 3 
and 4. 

We introduce the skew DTTs for the following reasons: 

• They are interesting from a signal processing point of 
view. As the DTTs, they provide a finite space model, 
their associated boundary conditions are simple, and their 
signal extension is 2-monomial (defined below) and also 
eventually periodic. 

• They are necessary building blocks in the Cooley-Tukey 
FFT type DTT algorithms that we will derive and present 
in detail in our next paper on the algebraic theory of 
signal processing, [14]. The special case of DCT type 
3 (and 2) algorithms and the first skew DCT were 
introduced in [13]. 

We follow our usual structure and derive first the signal 
model and then the associated Fourier transform. 



DCT-l„+i 



©C[x]/(a;-cos^) 



©C[a;]/(a;-cos^) 



Transposition of (114) yields a relationship between 
DCT-l„+i and DCT-3„. 



A. Building the Signal Model 

Shift, Linear Extension, Realization. The model we create 

DCT-2„ ® Ii is based on the finite space shift and thus its derivation follows 
the exact same steps as the derivation of the finite space model 
in Section VIII-A. The model we create now generaUzes the 
signal model for the DTTs in the T-group, i.e., the DCTs and 
DSTs of type 3 and 4 (see Table IX). The generahzation is 
done by modifying the right b.c.'s and thus the right signal 
extension as we explain next. 
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Boundary condition and signal extension. In the previous 
finite space model, we chose the right b.c.'s to ensure a 
monomial signal extension. Now, we just state the boundary 
conditions and derive the signal extension later. 

The goal is to modify the right b.c. of the DCTs and DSTs 
of type 3 and 4 such that the associated module is given by 
C[x]/{Tn — costtt), rGQ, 0<r<l. In this case we can 
read off the b.c.'s from Table IX: 



TABLE XIII 

(Right) signal extensions for the four skew DTTs. 



DCT-3„ 
DST-3„ 
DCT-4„ 
DST-4„ 



Tn = cosm, 
Un = Un-2 + 2cosr7r, 
Vn = -Vn-1 +2cosr7r, 
Wn = Wn-1 — 2cosr7r. 



(115) 



Thus, we obtain four equal modules C[x]/(T„ — costtt) with 
T-basis, J7-basis, F-basis, W-basis, respectively. 

In the general case r ^ 1/2, these b.c. lead to no monomial 
signal extension, since this property defines the signal models 
for the 16 DTTs. However, it is intriguing that the signal 
extension is "almost" monomial in the following sense. 



DCT-3n(r) 



DST-3„(r) 



DCT-4„(r-) 



DST-4„(r-) 





rt 2n— l,2n 3rt — l,3n 4n— l,4n 5n— 1 
n 2n— l,2n 3n— l,3n 4n— l,4n 5n— 1 




Definition 22 Using the notation from Definition 7, we call a 
signal extension 

Sk = '^0k,iSi, for A; 7, 

2-monomial if for each k the sum has at most 2 summands. 

Now, we can exphcitly state the signal extensions in these 
four cases. 

Lemma 23 The module C[a;]/(T„ — costtt) with T-, U-, V-, 
or W^-basis has a 2-monomial signal extension. More precisely, 
the signal extension is displayed in Table XTTT using the same 
method as in Figure 12 and Table X. The occurring constants 
are Ck = Tfe(cosr7r) = cos km and Uk = C/fc(cosr7r). The 
constants Uk scale the entire line they are adjacent to with the 
exception of the signal extension for the DCT-3 (r) (top fine), 
where the bottom values (bullets) are scaled by Ck- 

Proof: The proof uses induction and the two-term recur- 
rence of the Chebyshev polynomials. We show the induction 
step for the case of a F-basis in a boundary case. According 
to Table Xlll, 

Vkn-2 = Uk-lVn-2 ~ Uk-2Vl, 
Vkn-1 = Uk-lVn-1 - Uk^2Vo- 

We compute Vfe„ using the Chebyshev recurrence and the b.c. 
Vn = —Vn-1 — 2cosr7r, 

Vkn = 2a;Vfe„_i - Vkn-2 

= Uk-l{Vn-2+Vn)-Uk-2{Vo + Vi) 

-Uk-lVn-2 + Uk-2Vl 
= Uk-li-Vn-1 +2cOSrTTVQ) - Uk-2Vo 

= -Uk-iVn-1 + UkVa 

where we used 2 cos rn ■ Uk- 1 — Uk-2 = Uk, which is again the 
recurrence for Chebyshev polynomials. The result coincides 
with Table XIII, as desired. ■ 



Signal model: finite skew C-transform. Consider the 

regular module A4 = C[a;]/(T„ — costtt) with r E Q and 
< r < 1. Let C G {T,U,V,W}. The finite skew C- 
transform is the mapping 

$ : s ^ SkCk e 

0<k<n 

and is a signal model for V = C". As in the finite C- 
transform, the basis in the algebra is, independent of C, the T- 
basis: A= {h = J2o<k<n ^kTk}. For r = 1/2 or costtt = 0, 
the skew C-transform reduces to its ordinary counterpart. 

B. Spectrum and Fourier Transform 

To compute the spectrum and a Fourier transform for the 
regular module C[a;]/(T„ — costtt) with the four different 
bases, we need to determine the zeros of T„ — costtt and 
fix a proper ordering. 

Lemma 24 Let reQ, 0<r<l. We have the factorization 
T„-cosr7r = 2"-i JJ (a;_cos^7r), (116) 

0<i<n 

which determines the zeros of T„ — cos ttt. We order the zeros 
as a = (cosroTT, . . . ,cosr„_i7r), such that < rj < 1, and 
Ti < rj for i < j. The Ust a is given by the concatenation 

a= y (coS^TT.COsMiTT) 
0<i<n/2 

for n even, and by 

«=( U (cos 2:^77, cos Mi^r)) u (cos ^i^TT) 

0<i<S^ 

for n odd. In the particular case of r = 1/2 or cos rir = 0, we 
thus have a = (cos(i-|-l/2)7r/n | < z < n) as in Table XVI. 
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Proof: The zeros of T„ — cosrTr are proved using the 
closed form of T„. The ordering of a is shown by inspection. 
We omit the details. ■ 
In words, the list a arises from the list 7 = (cos(r + 2i)7r/n | 
< i < n) in (116) by interleaving the first half of 7 with 
the reversed (and reduced modulo tt) second half of 7. 

Lemma 24 gives also the spectrum of A^, which we will 
not state explicitly. Instead we now formally define the skew 
DTTs. 



with a denoting the zeros of T„ — costtt provided by 
Lemma 24. 

We do not explicitly state a convolution theorem, which can 
be obtained from Theorem 6. 

D. Translation into Non-Skew DTTs 

Each of the skew DTTs can be translated into their non- 
skew counterpart using a sparse x-shaped matrix. 



Definition 25 (Skew DTTs) Let p = Tn — cosrTr, < r < 1, 
and M. = C[x]/p with basis b = (Co, . . . , C„_i), where C 
is one of T,U,V,W. Let a = (cosri7r)o<i<ra denote the 
list of zeros of p in the order specified in Lemma 24. We 
denote the associated polynomial transforms Vb,a for M by 
DCT-3„ (r ) , mr^n (r) , DCT-4„ (r ) , DST:4„ (r) , for C = 
T, U, V, W, respectively. Further, we define for each of these 
four DTT(r) the associated scaled polynomial transforms 

DTT„(r) = diago<,<„(/(cosr,7r)) • DTT„(r), 

where / is the scaling function associated with (ordinary) 
DTT (see Table IX). We call these transforms skew DTTs. 
If r = 1/2, then DTT„(l/2) = DTT„ and DTT„(l/2) = 
DTT^ in all four cases. In the case of the DCT-3„(r) = 
DCT-3„(r), we will omit the bar for the skew versions. 
Equivalently we can define 

DCT-3„(r) = [cosrk£TT]()<k,i<n, 

DST-3„(r) = [sinrfc(^ + l)7r]o<fe,^<„, 

DCT-4„(r) = [cosrfe(£+l/2)7r]o<M<n, 

DST-4„(r) = [sinrfe(^+l/2)7r]o<fe/<„. 

As an example, we consider the DCT-43(l/3). Using 
Lemma 24, the zeros of T3 — cos(7r/3) = T3 — 1/2 are given 
by a = (cos(7r/9),cos(57r/9), cos(77r/9)). We get 



DCT-43(l/3) 



cos Y^TT 



cos jgTT 



C. Diagonalization Property 

The representation cf) afforded by C[a;]/(T„ — cosrTr) with 
the four C-bases, evaluated at the shift operator x, is obtained 
from (97) and (115). Thus, the shift matrix (f>{x) arises from 
(97) by adding in the upper right comer /3(r) = cosrTr for 
DCT-3(r), and /3(r) = 2 cos rTr for the other skew transforms. 
We obtain 



m = 2 



/3i 




/3(r) 



P3 

04 



(117) 



The values for the f3i coincide with the non-skew case given 
in Table XI. As a consequence, in the four cases, 

DTT(r) • (pix) ■ DTT(r)-^ = diag(a), 



Lemma 26 Let DTT,i(r) be a skew DTT. Then 
DTT„(r) = DTT„-X(*)(0, and 



DTT„(r) 



DTT„ ■X^:\r). 



Here, Xn\r) depends on the DTT and takes the following 
forms, indicated by * e {C3, 53, C4, 54}. 



Xp)(r) 



1 

ci 



Si 



Cl 









Sn-l 



Cn-1 
-Sn-l 



Cn-1 

c„ 



with = cos(l/2 — r)t-K/n and = sin(l/2 — r)£'jT/n. 



with 4 = cos(l/2 - r)(2^ + l)7r/(2r7,) and 4 = sin(l/2 - 
r)(2^+l)7r/(2n). For DST-4(r), the sines 4 in X^'^^'V) are 
multiplied by —1. 

In all four cases, if the lines intersect, the numbers are added 
at the intersecting position. 

Proof: Follows by direct computation, using the defi- 
nitions of the matrices and cos(a;) cos(?;) = (cos(x + y) + 
cos(x — y))/2. ■ 
The 2x2 blocks in the translation matrices Xn{r) are not 
rotations, which implies that the skew DTTs are not "almost" 
orthogonal in the sense of (102). However, using Lemma 26, 
we can easily invert skew DTTs by inverting X„(r). 

E. Translation between Skew DTTs 

All skew DTT(r) share the same associated module, but 
different bases. Thus they can be translated into each other by 
a base change similar to the ordinary DTTs in Section Vlll-H. 
As in that section, we consider the skew DCTs, type 3 and 4 
as an example. The base change matrix S'^ we computed in 
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(107) did not depend on the right b.c. Thus, the diagram (108) 
generaUzes, for arbitrary r, as 



C[a;]/(r„ -costtt) 



■C[a;]/(r„-cosr7r) 



DCT-3„(r) 



^C[x]/{x-ak) 



DCT-4„(r) 
■^C[x]/{x-ak) 



The first difference occurs when we extend (118) to the non- 
polynomial DCT-4„(r), since the scaUng diagonal depends on 

r. Let a = (ao, • • • , ctn-i) denote the zeros of T„ — costtt 
and / the scaling function of DCT-4 and let -Dn(r) = 
diago< 

k<n 

{f{ak)). Then 



C[x]/{Tn - cosrn) 



DCT-3„(r) 



S„ 



■C[x]/{Tn- cosrn) 



Dn{r) 



DCT-4„(r) 



■®C[x\/{x-ak) 



®C[x\/{x-ak) 

or, as an equation, 

D^{r) ■ DCT-3„(r) = DCT-4„(r) • S'^, (119) 

which generahzes (109). 

In Section Vlll-H, we continued by inverting (and trans- 
posing) this equation, using the fact that the DTTs are almost 
orthogonal, to derive the different relationship (113), which 
requires a smaller number of operations. As mentioned in 
Section IX-D, the skew DCTs are not "almost" orthogonal in 
the sense of (102). However, it is still desirable to invert (119), 
since we will need it later when we derive fast algorithms [14]. 
For this purpose we first define the proper "inverse" skew 
DTTs. The definition is motivated by and a generalization of 
the equations 

DCT-3-^ = 2/n • diag(l/2, 1, . . . , 1) • DCT-2„ 
DST-3-^ = 2/n-diag(l,l,...,l/2)-DST-2„ 



Thus, for r = 1/2, we have iDCT3„(l/2) = DCT-2„, 
iDST3„(l/2) = DST-2„, iDCT4„(l/2) = DCT-4„, 
iDCT4„(l/2) = DCT-4„. 

Note that Definition 27 does not provide direct knowledge 
about the matrix entries of the iDTTs. These, however, can 
be computed using Lemma 26. For example 

(120) 



iDCT-3„(r) 
iDCT-4„(r) 



■ DCT-2„, 
• DCT-4„, 



(118) and similarly for DST-3 and DST-4. Note that {xi*^( 



has in all four cases the same x-shaped pattern as X„{r). 
Namely, the four inverses are derived from 

-1 1 



cos a sin b 
sin a cos b 



For example. 



1 



cos(l/2 — r)7r 



cos(a -|- b) 



Cn 
C„_i 



-si 



cos 6 
— sin a 



— sin 6 
coso 





-Sn-l 



Cl 



Using Definition 27, we can now invert (109) to get a 
generaUzation of (111), 

Sn ■ iDCT-3„(r) • ^2?„(r)-i = iDCT-4(r)„, 
where Sn is given in (112). 

X. Alternative Infinite Space Model 

In the realization of the space model in (82), we set the 
space shift operator to q = x, which implied that the space 
marks were reahzed by the Chebyshev polynomials, t„ = Cn- 
This, in turn, produced a left boundary, and thus the need 
for left boundary conditions. Interestingly, and in contrast to 
the time model, it is possible to choose a different realization, 
which we will develop in this section. The visualization of this 
model will match Figure 15, which seems desirable, but we 
will encounter the different problem of collapsing frequency 
responses. 



and 

DTT"^ = n/2 • DTT^ = n/2 • DTT„ 
for DTT = DCT-4, DST-4. 

Definition 27 (Inverse Skew DTTs) We define the inverse 
skew DTTs by 

iDCT3„(r) = n/2-diag(2,l,...,l)-DCT-3„(r)-\ 
iDST3„(r) = n/2-diag(l,l,...,2)-DST-3„(r)-\ 

iDCT4„(r) = n/2-DCT-4„(r)-\ 
iDST3„(r) = n/2 • DST-4„(r)-^ 



A. Building the Signal Model 

Shift, linear extension, realization. We start with the same 
shift, the space shift, and the same linear extension as in 
Section Vll-A. In that section, the realization of the space 
model follows by setting the shift operator q = x, which 
impUed tn = Cn, the Chebyshev polynomials. Now we choose 
a different realization, namely by setting the space marks to 
tn = x" as in the time case. Necessarily, the shift now takes 
the form 



The corresponding fc-fold space shift can be obtained in two 
different ways. From (81) we can directly read off that qk = 
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TABLE XIV 

The different realizations in the space model and the 
alternative space model. 



space marks shift operator fc-fold shift operator 

space model C„ x Tk{x) 

alternative „ + x x~''+x'' 

space model 2 2 



{x~'' + x'')/2. Or, we use Lemma 9 to get qk = Tk{q) = 
{x~'' + x'^)/2 as can be seen from the power form of Tk in 
(152) in Appendix III. Table XIV contrasts the two different 
realizations of the abstract space model. 

As a result, we get as signal space M = {s = J^nei- ^nx"'}, 
and as filter space A — {h — "^^yo ^k{x~'' + x'')/2}. 
Filtering is well-defined for s G e^{Z) and h e £^{N). 

Signal model. The alternative infinite space model 
{A,A4,^) is given by the algebra of symmetric filters A = 
{h = J2k>o hk{x-^ + x^)/2} with h e £i(N) and the signal 
module M. consists of Laurent series s with s £ £^(Z). The 
mapping $ is, as in the time case, the .z-transform: 

s s = S{x) = ^ s„a;" e M. 

The important difference to the z-transform is the algebra, 
which is now smaller: it consists only of the symmetric filters. 
As an aside, this motivates why a signal model requires that 
we explicitly specify the algebra (see Definition 1). 

B. Spectrum and Fourier Transform 

The impact of a smaller algebra when compared with the 
time case becomes evident when we compute the spectrum, 
which we do next. Since the set of operating filters, the 
symmetric filters, is smaller than in the time case, every 
eigenfunction iJ^j [x) in the time case is also an eigenfunction 
in the space model under consideration. This implies that the 
Fourier transform for M. can be chosen as in the time case 

A: s = 5(a;)^(5(e^-))^e[o,2.). 

Accordingly, the frequency response of the filters h = 
H[{x~^+x) /2) e ^ at frequency w, or, in algebraic terms, the 
irreducible representation afforded by the one-dimensional 
module generated by E,^ [x) becomes 

(j)^ : H{{x~'^ + x)/2) ^ H{cosjuj). 

This shows that pairs of conjugate frequencies ijJ,lO afford the 
same representation 

4>oj = (h, 

i.e., produce the same frequency response. As we explain 
later, this property of collapsing frequency responses carries 
over to the finite-dimensional case, where it may serve as 
an explanation why certain transforms work in practice better 
than others on "space signals," e.g., images. Further, it shows 
that there is a larger degree of freedom in choosing a Fourier 
transform, since in any of the two-dimensional eigenspaces 
spanned by E^{x), Eijj{x) we can choose any basis. 



C. Visualization 

The visualization of the associated signal model is given 
in Figure 15, i.e., no boundary is intrinsic to this model and 
thus it seems more natural at first glance. However, this model 
has the different problem of collapsing frequency responses as 
explained in the previous section. 

D. Remarks 

As a summary, we observe that different realizations of 
the same abstract model may be possible, which motivates 
the concept of realization. Different realizations may have 
different properties and shortcomings. For example, in the 
case just studied, we trade the need for a left boundary by 
the collapsing of conjugate frequencies. We have no proof 
that these two realizations of the space model we considered 
are the only ones possible. However, regardless of the chosen 
realization, the Chebyshev polynomials come into play as a 
consequence of Lenmia 9. 

XI. Alternative Finite Space Model: RDFTs and 
DHTS 

In Section X, we discussed an alternative realization of the 
infinite space model. In contrast to the original realization 
of the infinite space model presented in Section VII, the 
alternative realization does not require a left boundary, but 
has the counterpart of collapsing frequencies, i.e., the spectrum 
consists of two-dimensional eigenspaces. As a consequence, 
the Fourier transform was no longer uniquely determined since 
in each of these two-dimensional spaces any basis can be 
chosen. 

In this section, we discuss this alternative realization briefly 
in the finite-dimensional case and derive the corresponding 
transforms. We will see that, similar to the infinite case, there 
will be the problem of collapsing frequencies and thus a larger 
degree of freedom in choosing the Fourier transform. This 
degree of freedom interestingly leads, for different reasons, 
practically to the same class of transforms that arose as real 
Fourier transforms for the finite time model in Section VI-G. 

A. Building the Signal Model 

Shift, linear extension, realization. The altemative real- 
ization of the signal model in Section X led, as in the time 
case, to the ^-transform. Thus, the corresponding finite signal 
modules also coincide with the finite time case, namely, they 
are given by €,[x\/p{x) with basis h = {x^ ,x^ , . . . ,x^~^). 
The question of the monomial signal extension in this case 
has already been settled in Lemma 8, namely by requiring 
p(x) = x" — a, where a G C, a 0, is any complex constant. 
As an aside, this emphasizes that the signal extension is a 
property of the module (including the chosen basis) only, and 
independent of the chosen operating algebra. 

The difference to its time counterpart lies in the operat- 
ing algebra of filters; the modules are in both cases equal, 
namely M = C[x]/{x" — a). In the time model, this module 
was regular, A = A4, since A contained the time shift 
operator q = x, which generates the entire algebra A. In 
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the alternative space model, however, the shift operator is 
given hy q = {x^^ + x)/2; see (121). The algebra for the 
space model may now, as in the infinite case, be smaller than 
C[x]/{x"' — a), namely consisting only of those filters that 
are expressible as a polynomial in q. In algebraic terms, the 
algebra for the alternative finite space model is the subalgebra 
of C[x]/{x" — a) generated by q: 

A = ((a;-^ + x)/2) < C[x]/{x" - a). 

Note that is well-defined in C[x]/ (.r" — a) (since a ^ 0) 
and given by = a;"~^/a. The question now is whether A 
is equal to or smaller than C[x]/{x"' — a), in particular, for 
the most interesting case a = 1. As we show next, the answer 
depends on a, and, in the case where A is smaller, it is an 
algebra belonging to a DCT 

Lemma 28 (Algebras for the alternative finite space model) 
The subalgebra A = ((a;"^ +a;)/2) of C[x]/{x"- - a) is equal 
to C[x]/{x" - a) if and only if a 7^ ±1. If a = ±1, then the 
structure of A is given by: 

'C[x]/{Trn ' T„,_2), m = f + 1, n even, a = 1 

^ ^ C[x]/T„i, m — ^ + 1, n even, a = — 1 

" I C[a;]/(T„ - T„_i), m=^^, n odd, a=l ' 

C[x]/{Tm + Tm-i), m = 2±1, n odd, a = -1 

which are the algebras associated to the DCTs of types 1,3,5,7, 
respectively. The number m is their respective dimension. 

Proof: A straightforward way to compute the algebra 
is to consider the sequence of powers l,q,q^, . . ., and to 
determine when they become linearly dependent. We choose 
a different way by considering the sequence of A:-fold space 
shift operators (the natural basis in the space model) Ti.{q) = 
{x~^ +x^)/2, fc > 0, and focus on the case where n is even. 
Using x~'' = x"~''/a, we get the following n elements in 
C[a;]/(a;" - a): 

lx"-^+X i;r"-2+;j;2 
1 _a _a 

' 2 ' 2 

lj.n/2+1 , j.n/2-1 ^ i i l^n/2-1 , W2+1 
a a a 

2 ' 2 ' 2 

a 

2 

The next element would be (l/a + a)/2, which makes the set 
certainly hnearly dependent (also due to the length exceeding 
n). The question is when this set is linearly independent. We 
observe that the powers x'' and a;""*^ always occur together. 
Thus the set is hnearly dependent if and only if there is an 
a G C such that for any k, 

-x'^-^+x^ = a{x^ + -x''-^) 
a a 
<^ a = ±1. 

Conversely, if a = ±1, then only the first n/2 + 1 polynomials 
To((j'), Ti(g), . . . , Tn/2{q) are hnearly independent. We focus 
on the case a = 1 in which T'„/2+i(9) = Tnj2-i{q). This 
shows that 

A^€.[q]/{Tm{q)-Tm-2{q)), m = n/2 + 1, 



as desired. From Table IX, we see that this algebra is associ- 
ated to the DCT-1„. The cases a = — 1 and n odd are derived 
analogously. ■ 
Signal model. In summary, we obtain a signal model for 
F = C" given by 

$ : s 1-^ ^ Skx'' e M, 

0<k<n 

with module M = C[a;]/(a;" — a), and A is given by 
Lemma 28. The natural basis in A consists of the fc-fold space 
shift operators Tfc((a;-^ -|- a;)/2) = (a;-'= -|- a;'=)/2. 

B. Spectrum and Fourier transform 

Lemma 28 shows that for a = ±1, the algebra A is 
of smaller dimension than the module, namely n/2 + 1 (or 
(n + l)/2). Because of Lemma 3, A has only n/2 + 1 different 
irreducible representations. Thus, as in the infinite case, we 
have the problem of collapsing frequencies: the n spectral 
components (eigenspaces) of M produce only n/2 -I- 1 (or 
(n-h l)/2) different frequency responses (irreducible represen- 
tations). More precisely, focusing on a = 1, decomposes 
as in the time case (regular case) as 

C[a;]/(a;" - 1) ^ C[a;]/(a; - • • • C[x\/{x - 

We set Mk = C[x]/(x — tj^). The irreducible modules M.k 
are mutually different (as vector spaces), but their frequency 
responses are not. Namely, because of 

(a;""^ -h x)/2 • a:° = cos2fc7r/n mod (a; - w^) 

the spectral components Mk and Mn-k afford the same 
irreducible representation = i.e., produce the same 

frequency response. In other words, conjugate frequencies in 
the (regular) time model collapse to the same frequency if 
is viewed only as an ^-module, A < C[a;]/(a;" — 1), and not 
as a regular module. 

As a consequence of the previous section, there is a larger 
degree of freedom in choosing a Fourier transform for the A- 
module A4. In the time model, the generic Fourier transform 
for the regular module A4 was given by 

JT = D • DFT„, 

where D was the degree of freedom: any invertible diagonal 
determined by the choice of bases in the irreducible modules 
A^fc. In the present case, the degree of freedom is in choosing 
bases in the two-dimensional spaces Mk ® Mn-k, which 
afford the same frequency response, and the remaining one- 
dimensional spaces A^o and A^„/2 (if n is even). Regarding 
the Fourier transform, this leads to a similar situation as in 
Section VI-G, but for different reasons. In Section VI-G, 
we reduced the base field, which caused pairs of conjugate 
spectral components to fuse to real spectral components of 
dimension 2. In the present case, the base field is still C, 
but the smaller algebra causes pairs of conjugate frequency 
responses to become equal, i.e., the spectral components still 
have all dimension 1, but afford the same representation. 
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As a result, the generic Fourier transform is given by any 
matrix of the form 



T = X- DFT„ 



(122) 



where X is any invertible matrix of the x-shaped form (70). 
The difference between (122) and (69) is that every matrix in 
(69) is one in (122) but not vice- versa, since the matrix in (69) 
has to be real valued. As a consequence, the RDFT and the 
DHT, defined in Section Vl-G, are Fourier transforms for the 
finite alternative space model. 

C. Visualization 

We visualize the alternative finite space model for a = ±1 
using Definition 2 by the graph in Figure 19 that has the shift 
matrix (123) below as adjacency matrix. The graph is similar 
to Figure 13 but is undirected, i.e., all edges are now in both 
directions. If a ^ ±1, then the edge from a;"~^ to has 
weight a, while the reverse edge has weight 1/a. 



X 

- • 



-n— 1 



-2 I 



Fig. 19. Visualization of the alternative space model for a = ±1. 



D. Diagonalization Properties 

Let (p be the representation afforded by the ^-module M. = 
C[a;]/ (x" — 1) and consider the shift operator q = {x~^+x)/2. 
It is 

1 


1 



m = I ■ 



(123) 



which should be compared to (97); the inner structure is 
the same, determined by the space shift, but the boundary 
conditions are different. Further, (f>{q) and thus the signal 
model is symmetric (see Definition 18) if and only if 
a = ±l. 

From the above discussion, we know already that, for a = 
±1, (j){q) has duplicate eigenvalues in contrast to the matrix 
in (97). In particular, for a = 1, we get the diagonaUzation 
property 

rm^'^ = diag(l, cos ^, cos f , cos f , cos f ), 



where the value in the middle of the diagonal depends on 
whether n is even or odd. In this equation, can be chosen, 
e.g., as DFT„, RDFT„, DHT„. The generic matrix diagonal- 
ized by this transform is given by h G A. The structure 
of (j){h) for a = 1 is symmetric circulant. 

As a summary, the RDFT and the DHT diagonalize sym- 
metric circulant matrices and "almost" diagonalize (i.e., reduce 
to x-shape) circulant matrices, shown in (76). 

XII. The Generic Nearest Neighbor Model 

Signal models based on the time shift operator (51) and, 
as we have shown, the space shift operator (80) are widely 
used in signal processing, reflected by the common use of 
the DFT and the DTTs. After recognizing that the concept 
of a shift is by no means restricted to the time shift used in 
classical signal processing theory, it is natural to start exploring 
other shift operators, their associated infinite and finite signal 
models, and their use in signal processing. In this section we 
briefly discuss a generaUzation of the space shift: the generic 
nearest neighbor (GNN) shift^*". Based on this shift, we first 
derive the infinite and the finite signal model, following the 
same steps as used before in the derivation of the time and 
space models. 

A. Building the Infinite GNN Signal Model 

Definition of tlie GNN sliift. As before, we denote the 
abstract shift operator by q and define the generic nearest 
neighbor shift by 



(124) 



which is depicted in Figure 20. 

We require that a„, c„ ^ 0. As usual, the t„ denotes "space 
marks''^''. Note that, in contrast to the time and space shift, 
(124) is variant since the coefficients depend on n. 



qo 



tn-1 



in 



tn+1 



Fig. 20. The generic nearest neighbor shift qot„. 

In this generic case, it is not clear how to define a fc-fold 
GNN shift. A natural choice would be a polynomial „ of 
de gree k such that Pk,n{Q)tn is a linear combination of only 
tn-k and tn+k (the marks at distance k from t„). However, 
this is not possible in the general case as can be shown by 
direct computation (compute i„, got„, g^ot„, g^ot„; it is not 
possible to linearly combine them so that the result has only 
t„_fe and as summands). 

Linear extension. As usual, we linearly extend the oper- 
ation of q to the entire set A4 = {s = X^„gzSnin}- Since 

^*We could call it the Markov shift for obvious reasons and call the resulting 
model a Markov model. Since this term is already taken, we refrain from doing 
so. We will discuss the relationship between the GNN model and Gauss- 
Markov random fields in Section XVI. 

^^The word "space" is used in lack of a better term; the model is only 
"space" in the sense that the shift connects to both neighbors. 
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we do not have a notion of fc-fold GNN shifts, it is not clear 
which basis to choose in the associated algebra of filters. 

Realization: ortliogonal polynomials. We realize the 
model by setting q = x, which leads to the recurrence 

X — bn p 



n+1 



L P 



1) 



(125) 



which also motivates the requirement c„ ^ 0. Normalizing 
Po = 1 and dcg(Pi) = 1 defines a sequence of polynomials 
orthogonal with respect to some moment functional (Favard's 
theorem), [87, p. 21]. Conversely, every sequence of orthog- 
onal polynomials obeys a recurrence of the form (125) [87J. 
The recursion (125) can be run in the negative direction (since 
we required a„ ^ 0) to obtain the P„ for n < 0. Every such 
Pn is again a polynomial and can thus be expressed as a linear 
combination in the P„, n > 0. As a consequence, we consider 
only right-sided sequences, and M ~ {s = '^n>o^nPn}- 
Consequently, A also contains only right-sided sequences. 

The sequence P„, n < is the left signal extension, and 
expressing P_i in Po, Pi is the left b.c. 

Signal model. As a result, we obtain an infinite discrete 
signal model for a vector space c given by the P- 
transform 

$ : s = (so, si, . . . ) ^ SfePfe. 

fe>0 

Choosing V = ensures that "I>(s) defines a function on 

the interval of orthogonality of the P„. How much V can be 
enlarged depends on P, see [60]. The algebra of filters is a set 
of right sided series, but as said above, it is not clear which 
basis polynomials to choose. 

Finally, we conjecture that it is not possible to establish 
monomial left b.c. unless P^ e {Tk,Uk,Vk,Wk}- Note that 
Pfc = x'' cannot be included here since we required a„ ^ 
in the recurrence (125). 

We did not pursue the question of the exact form of the 
Fourier transform, and we also omit the visualization of the 
P-transform. 

B. Building the Finite GNN Signal Model 

Shift, linear extension, realization. As before a straight- 
forward realization leads to the set of all polynomials 



Sfe-Pfe, 



0<fe<n 

which is the vector space C„[.t], but not a module, since 
xPn-i ^ Cn[x]. To solve this problem we need a right b.c. 

Bomidary condition and signal extension. We can choose 
any right b.c. P„ = 'Yl,a<<k P^Pk, but the relevant question 
is which one is desirable. We cannot expect to establish 
a monomial signal extension as in the special case of the 
Chebyshev polynomials. 

We argue that a natural choice is the b.c. P„ = 0, since it 
achieves three crucial properties. 

• P„ is separable, a general property of orthogonal polyno- 
mials [85, p. 44], which ensures one-dimensional spectral 
components. 

• The associated Fourier transform is "almost" orthogonal. 



• The Fourier transform has a fast algorithm. 
The last two points are discussed in greater detail below. 

In summary, we obtain the regular module M. = 
C[x]/Pn{x) with P-basis. 

Signal model. The corresponding signal model is given by 
the finite P-transfonn 

$ : S 1-^ ^ SfePfc e C[x]/Pn{x), 
0<k<n 

where M = C[x]/Pn{x) is regular, i.e., A = M. We call this 
signal model also the finite GNN model. 

C. Spectrum and Fourier Transform 

We obtain the (polynomial) Fourier transform for the regular 

module <C[x]/Pn{x) with basis b = (Pq, . . . , P„_i) as a 
special case of (41) and Theorem 4. We assume that a = 
(ao, • • • , ctn-i) are the (mutually distinct as mentioned above) 
zeros of P„. Thus, the Fourier transform is given by 

A: C[x]/P„Xx) ^ 0^,^^^^^C[x]/(x-a,), ^^^6) 
s = s{x) (■s(ao), • • • , s(q!„_i)). 

In matrix form, we get the polynomial Fourier transform 



^ = 'Pb,a = [Pe{o:i)]o<k,e<n- 



(127) 



D. Diagonalization Properties 

If (p is the representation of A afforded by M., then T 
diagonalizes any matrix h & A. In particular, the shift 

matrix (j){x) is tridiagonal: 



6o ai 

Co bi 

Cl 



02 
C2 



an-2 
bn-2 

Cn-2 



flri-l 
bn-l 



(128) 



where 6o,co depend on the left b.c, or equivalently on Pi. 
Thus, 

T(l>{x)T~'^ = diag(ao, . . . , a„_i). 

A convolution theorem follows directly from Theorem 6 and 
is not explicitly stated. 

E. Orthogonal Transform 

The (polynomial) Fourier transform T for the regular mod- 
ule M = C[x]/Pn with basis b = (Pq, . . . ,P„-i) in (127) 
is almost orthogonal in the sense that there exist diagonal 
matrices D, E such that DTE is orthogonal. We showed 
this general result before in Theorem 16 and used it to 
derive the matrices D, E to obtain the orthogonal DTTs. We 
further showed that if a DTT is a Fourier transform for a 
signal model then the corresponding orthogonal DTT is 
a Fourier transform for a slightly adjusted signal model 
(Theorem 17), which arises from $ by a scaling of the module 
basis, i.e., a base change with a diagonal matrix E. 

The derivation for the DTTs in Section VIII-F transfers di- 
rectly to the finite GNN model. Let T be the Fourier transform 
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in (127) for the finite GNN model {A,M,^) with shift matrix 
4>{x) in (128). Further, let D, E be diagonal matrices computed 
using the Christoffel-Darboux formula (Theorem 15), to make 

orthogonal {E~^ is chosen to obtain the form in equa- 
tion (104)). Then, E(j){x)E-'^ is symmetric. Further, if E = 
diag(eo, . . . ,e„_i), then J^' is a Fourier transform for the 
signal model {A,M,<^'), with 

: s Sk—Pk- 

0<k<n 

This model is symmetric (see Definition 18), since, if (p' is 
its afforded representation, then the matrix 4>'{x), and thus all 
matrices (t>'{h), /i G ^, are symmetric. 

As we mentioned before, this general construction of or- 
thogonal transforms from orthogonal polynomials has been 
proposed in signal processing by [86], where it was called 
Gauss- Jacobi procedure and the resulting transforms Gauss- 
Jacobi transforms. Our theory identifies these transforms as 
Fourier transforms for finite GNN signal models and provides 
the associated filters, spectrum, boundary conditions, signal 
extensions, and other concepts and properties. 

A few special cases, corresponding to special choices of 
orthogonal polynomials, have been proposed in the signal 
processing literature including for the Laguerre polynomials 
[91], for the Hermite polynomials [92], and for the so-called 
discrete orthogonal polynomials [93], [94], [95]. 

With our algebraic theory, we get immediately all important 
concepts associated to these transform including an under- 
standing of the underlying signal model, the notion of filtering, 
and several properties of the transforms. 

F. Fast Algorithms 

Although fast algorithms are not the subject of this paper, 
and will be considered in [14], we want to mention that every 
polynomial transform constructed as above from orthogonal 
polynomials has a fast algorithm that allows its computation 
using only 0(nlog^(n)) operations (see [63], or the numeri- 
cally more stable version in [64]). These algorithms are based 
on the three-term recurrence characteristic for orthogonal 
polynomials. Note that this cost is sUghtly worse than for 
the special case of the DTTs, which are known to have a 
complexity of 0(nlog(n)). This implies that DTT algorithms 
are due to special properties of the Chebyshev polynomials, a 
fact that we will confirm in a different paper, [14], in which 
we extend our algebraic theory to the derivation and discovery 
of fast algorithms. 

XIII. Overview of Finite Signal Models 

In Table XV we list all the finite signal models, and their 
associated Fourier transforms, that we introduced in this paper. 
The table is divided into complex time models, real time 
models, complex/real space models (in contrast to time, for 
space the restriction to a real base field does not change the 
spectrum or the Fourier transform), and complex/real GNN 
models. In each row, we list in the first three columns the 



signal model as {^,M.,A), in the fourth column the associated 
unique polynomial Fourier transform, and in the fifth column 
possibly other relevant Fourier transforms for the model. Note 
that the notion of polynomial transform only exists for regular 
modules of separable polynomial algebras; thus, the real time 
models and the alternative space models have no polynomials 
transforms (in the table indicated as n.a.). 

Orthogonal transforms. Each of the listed transforms has 
an orthogonal counterpart, which is in each case obtained by 
proper scaling of rows or columns. In some cases, this scahng 
requires an adjustment (namely a scaling of the module basis) 
of the signal model. This is the case for certain DCTs/DSTs 
(Section Vlll-F) and for the GNN transforms (Section XII-E) 
based on general orthogonal polynomials. 

Discrete trigonometric transforms. In this paper, we have 
used the term discrete trigonometric transforms (DTTs) to 
denote the 16 DCTs and DSTs. In the literature, the DTTs are 
often considered as the entire class of transforms whose entries 
are expressible using cosines and sines, which includes also 
the DFT, DHT, and RDFT To our best knowledge. Table XV 
contains all 1-D trigonometric transforms (in this sense) that 
have been introduced in the literature, and extends this class by 
the RDFTs of types 2-4, the polynomial DCTs and DSTs, and 
the four types of skew DCTs and DSTs. Further, we suggest to 
rename the (rarely occurring) DWTs of type to be called 
DHTs of type 1-4. 

The Fourier transforms for the GNN model are not trigono- 
metric transforms and should not be considered as such. 



XIV. Higher-Dimensional Signal Models 

In Section Il-D, we identified the equivalence (under some 
weak assumptions) of signal models with shift-invariant sys- 
tems (or filters) and commutative algebras A. More specifi- 
cally, we asserted that if A consists exclusively of FIR filters, 
then A is necessarily a polynomial algebra, i.e., of the form 

A = C[x]/{px{x),...,Pk{x)) (129) 

where x = (xi, . . . ,Xrn) and the pi are polynomials in n 
variables. Usually, the generators of this algebra are chosen as 
xi, . . . ,Xn, which are the shift operators in this algebra. The 
whole discussion in this paper focused on the case of only one 
shift operator x = xi, which corresponds to signal models for 
sampled 1-D signals. In this case, A = C[x]/p{x), which, as 
a regular module, provides the underlying signal model for 
most 1-D transforms including the DFT and the DTTs. 

In signal processing, m-D signals are usually processed with 
TO-D versions of 1-D transforms. In matrix form, if T is some 
1-D transform, such as the DFT or a DCT, its m-D counterpart 
is simply the m-fold tensor (or Kronecker) product of .F with 
itself, 

O.-.OJ^. (130) 

The resulting transform is called "separable," since it operates 
independently along the different dimensions, according to the 
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TABLE XV 

Overview of all finite signal models and associated Fourier transforms discussed in this paper. All these Fourier transforms 

(EXCEPT those for THE GNN MODEL) ARE TRIGONOMETRIC TRANSFORMS. 



Time (complex): complex finite z-transform 






Section VI-B 




M 


A 




other T 




C[x]/{x'' - a) 


regular 


DFT„ D 


— 




C[x]/(x"- - 1) 


regular 


DFT„ = DFT-1„ 


DFT-2„ 




C[x]/(x" + 1) 


regular 


DFT-3„ 


DFT-4„ 


Time (real): real finite z-transform 






Section VI-G 




M 


A 


^ = n,a 


Other T 




R[x]/(x" - 1) 


regular 


n.a. 


RDFT„ = RDFT-1„ 




R[a;]/(a;" - 1) 


regular 


n.a. 


RDFT-2„ 




R[x]/{x" - 1) 


regular 


n.a. 


DHT„ = DHT-U (DWT-ln) 




]R[x-]/(x-" - 1) 


regular 


n.a. 


DHT-2„ (DWT-2„) 




IR[x-]/(x-" + 1) 


regular 


n.a. 


RDFT-3n 




R[x]/{x" + 1) 


regular 


n.a. 


RDFT-4„ 




IR[x]/(x" + 1) 


regular 


n.a. 


DHT-3,, (DWT-3„) 




L?k]/(.r" + 1) 


regular 


n.a. 


DliT-4„ (i)\\"T-4„ ) 


Space (complex/real): finite C-transform (C = 


T,U,V,W) 




Sections VIU-B, IX-B, XI-B 




M 


A 




other T 




C[x]/{x^ - l)U„-2 


regular 


DCT-1„ = DCT-1„ 






C[x]/T„ 


regular 


DCT-3„ = DCT-3„ 






C[x]/(x- 


regular 


DCT-5„ = DCT-5„ 






C[x]/ix+l)V„-i 


regular 


DCT-7„ = DCT-7„ 






C[x]/iT„ - cosr-Tr) 


regular 


DCT-3^(r) = DCT-3^(r) 






C[x]/T„ 


regular 


DST-3n 


DST-3„ 




C[x]/U„ 


regular 


DST-1„ 


DST-1„ 




C[x]/V„ 


regular 


DCT-7„ 


DCT-7„ 




C[x]/W„ 


regular 


DST-5„ 


DST-5„ 




C[x]/{Tn -cosrTr) 


regular 






s i-» E Sfc Vfc 


C[x]/{x - l)W„_i 


regular 


DCT-6„ 


DCT-6„ 




C[x]/Vn 


regular 


DCT-8„ 


DCT-8„ 




Clx]/{x - l)!7„_i 


regular 


DCT-2„ 


DCT-2„ 




C[x]/Tn 


regular 


DCT-4„ 


DCT-4„ 




C[x]/{Tn - cosr-Tr) 


regular 


DCT-4(r).n 






C[x]/(a; + l)Vn-i 


regular 


DST-8n 


DST-8„ 




C[x]/Wn 


regular 


DST-6n 


DST-6n 




C[x]/Tn 


regular 


DCT-4„ 


DCT-4„ 




C[x]/{x+l)Un^l 


regular 


DST-2„ 


DST-2„ 




C[x]/(Tn - cosrTr) 


regular 


DST-4(r)„ 


DST-4(r)n 


s 1-^ 'E.Skx'' 


C[x]/{x" - 1) 


({x-'^+x)/2) 


n.a. 


RDFT„ = RDFT-1„ 




C[x]/(x" - 1) 


{(x-l +x)/2> 


n.a. 


RDFT-2„ 




C[x]/(x" - 1) 


((x-l +x;)/2> 


n.a. 


DHT„ = DHT-U 




C[x]/(x" - 1) 


((x;-l +rr)/2> 


n.a. 


DHT-2„ 




C[x]/{x'' + 1) 


((i-i +a;)/2> 


n.a. 


RDFT-3n 




C[x]/{x" + 1) 


{ix-^+x)/2) 


n.a. 


RDFT-4n 




C[x]/(x" + 1) 


{(i-l +x)/2> 


n.a. 


DHT-3„ 




C[a;]/(x" + 1) 


{(i-i +a;)/2> 


n.a. 


DHT-4„ 


GNN (complex/real): finite P-transform (P orthogonal polynomials) 


Section XH-C 


<S> 


M 


A 


^ = n,a 


other 


s ^ E SkPk 


C[x]/Pn 


regular 


no specific name 
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fonnula 

This also shows that the m-D version of T can be computed 
using mn™~^ many 1-D ^'s. The question is: what is the 
underlying signal model, i.e., how to represent m-D transforms 
in our algebraic theory? Clearly, the signal model is shift- 
invariant, which impUes that the algebra has to be of the form 
(129). 

In this section, we explain that algebras for separable signal 
models are a special case of (129), and that the algebraic 
construction of a separable m-D signal model from its 1-D 
counterpart is a natural one. We focus our discussion on the 
2-D case; the m-D case is completely analogous. 

Separable module and algebra. Let Mx = C[x]/p{x) be 
a regular module with basis = {po{x), . . . ,Pn-i{x)). The 
a; in the subscript is the name of the variable or shift operator; 
we mention this since we will create copies with different 
variable names. Then, a corresponding regular module in two 
variables can be constructed as 

Mx.y = Ax.,y = C[x,y]/{p{x),p{y)). (131) 

Note that this is a special case of (129) for m = 2, and, 
more important, the two polynomials p{x),p{y) depend on 
different variables; this is equivalent to the separability. The 
natural basis hx,y for M.x,y is the cross product of the basis 
hx with by, namely 

bx.y — bx X by 

{po{x)po{y),po{x)pi{y), . . . ,po{x)pn-i{y), 
_ Pi {x)po {y),Pi {x)pi {y),...,pi {x)pn-i (y) , 



Pn-lix)po{y),Pn-l{x)pi{y), . . . ,Pn-l{x)pn-l{y)), 

(132) 

which shows that the dimensions multiply, i.e., Mx,y has 
dimension n • n = n?. Omitting one of the polynomials in 
(131), say p{y), would make the module infinite-dimensional, 
since it would now contain any polynomial in y. Note that 
two shift operators operate on the above basis: x and y, the 
generators of Ax.y 

Signal model. The regular module Mx,y provides a signal 
model for C" , whose elements we assume to be arranged in 
a two-dimensional array s = {sk,e)'- 

$ : s ^ Sk,mPe- 

0<k<n 

o<e<n 

Decomposition into a tensor product. Instead of working 
with (131), it is more convenient to exhibit the separable 
structure of A4x,y, which is encapsulated algebraically by 
decomposing the module into a tensor product: 

Mx,y = Mx O My, 



with associated algebra Ax,y = Ax (8) Ay. The elements of 
a tensor product of vector spaces are generated by its natural 
basis bx,y in (132). Intuitively, the tensor decomposition de- 
couples the two variables and makes explicit the separability. 
Since the tensor product is a "natural" construction in algebra, 
most relevant properties are also "naturally" derived from the 
1-D counterpart. 

Spectrum and Fourier transform. Let p{x) = 
no<fc<ra(^~"fe)- "^^^ Fourier transform of the regular module 
Mx X My is given by 

A : C[x]/p{x) X C[y]/p{y) ^ 

C[x]/ix-ak,)xC[y]/iy-ak,), (133) 

0<ki <n 
0<)s2<" 

and for every signal the spectrum is computed as 

s = s{x,y) A(s) = {s{ak,ae))o<k,e<n, 

using s{x, y) = s{ak, a£)mod(a; — ak,x ~ ai). 

Note that the irreducible modules in (133) have all dimen- 
sion l-l = l.If.Fisa Fourier transform for Mx with basis 
bx, then T ® T is & Fourier transform for Mx x My with 
basis bx x by. 

Representation. We evaluate the representation (()x^y af- 
forded by the regular module Mx.y with basis bx.y at the 
shift operators x and y. The operation of x on bx yields the 
matrix (t)x{x). Thus, from the special form of bx,y in (132), 
we obtain 

(t>x,y{x) = (t>x{x) ® I„, 

and, analogously, 

4>x,y{y) = \n®4>y{y)- 

As usual, and obvious in this case, T ® T diagonaUzes both 
matrices. 

Visualization. A visuaUzation of the separable signal model 
is obtained by constructing the graph with adjacency matrix 
^x,y{x) + (l>x,y{y) = (f>x,yix + y), which is exactly the 
direct product of the graph for the 1-D model given by the 
adjacency matrix (t>x{x) with itself. For example, for the model 
underlying the two-dimensional DCT of type 2 and size 8, 
which is used in the JPEG image compression standard, we 
obtain this way Figure 21 from the second graph in Figure 17. 
AH horizontal lines are incurred by the a;-shift, all vertical 
Unes by the y-shift. The loops at the boundary visuaUze the 
symmetric b.c.'s of the DCT, type 2. 

For the DFT, the direct product of the directed circle graph 
(Figure 13) with itself is a directed torus. 

Summary on separability. As a summary, it is worth to 
emphasize again that the separable construction of m-D signal 
models is quite natural, given the equivalence of the following 
concepts: 

tensor product of algebras/regular modules 
^ tensor product of Fourier transforms 
direct product of visuaUzing graphs. 

Non-separable signal models. Separable signal models are 
commonly used in signal processing for two main reasons: 
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Fig. 21. Visualization of the 2-D space model corresponding to the DCT, 
type 2. 

• They produce models for square grids, which is by far the 
most common format for m-D signals such as images. 

• The associated Fourier transforms and its fast algorithms 
are immediately available from the 1-D counterparts. 

An interesting question is whether there are any interesting 
non-separable signal models. The above discussion shows that 
they would likely model a non-square grid. Indeed, using the 
algebraic theory, reference [3] derives a non-separable 2-D 
transform, called discrete triangle transform, that operates on 
a regular triangular grid. Further, also using algebraic methods, 
it can be shown that this transform possesses fast algorithms 
[15]. A more detailed paper is in preparation. 

Remarks. In the case of one variable, or shift operator, 
we have argued before that the algebra has always the form 
C[x\/p{x). This was due to C[x] being a Euclidean ring, which 
allows us to always reduce 0^x\/ {pi{x),p2{x)) to <C[x\/p{x), 
with p = gcd(pi,p2)- In the m-D case (129) with more than 
one variable m > 1, this is in general no longer the case, i.e., 
the number k of polynomials "factored out" in (129) can be 
arbitrarily large. If fc < m, then the dimension of A is infinite; 
if fc > m the dimension depends on the choice of he pi. The 
investigation of these multivariate polynomial algebras is the 
subject of the theory of varieties [62]. 

XV. Markov Models, Graphs, Matrices, and 
Signal Models 

The construction of the time model, the space model, and 
the generic nearest neighbor (GNN) model in Sections V, VII, 
and XII, respectively, consisted of two essential steps: 

1) The introduction of (time/space) marks t„, the abstract 
shift operator q, and its operation on the tn- 

2) Extension to Unear combinations of marks on the one 
hand to form the signal module, and to linear combina- 
tions of multiple shift operators on the other hand to form 
the filter algebra. 

The second step is a common mathematical procedure and the 
effect, of course, is that we obtain a linear signal model. The 
question remains to identify the mathematical nature of the 
first step, and we will give a partial answer by relating this 
step, under certain assumptions, to the definition of a discrete 
Markov chain. Afterwards, we will discuss the general rela- 
tionship between the concepts of finite- state Markov chains, 
weighted graphs, signal models, and square matrices. 

A. Discrete Markov Chains 

We start with the definition of a discrete Markov chain, 
following [96]. 



Definition. We assume a system S" of a discrete number of 
mutually exclusive states, abstractly denoted by n € / C Z. 
At an initial instant of time, fc = 0, the uncertainty of the 
state is described by a random variable X^, whose distribution 
(«o,ra)ne/ describes the probabiUty of S being in state tn, 
n G I. The system is now observed at regular time instances 
fc e N and the state distribution evolves according to a fixed 
stochastic rule. Namely, if at time fc, the system is in state t„, 
then the probability that at time fc-|- 1 the system is in state tm 
is Pm,n, independent of the states at previous time instances 
0,...',k-l: 

Pm,n = prob(Xfe+i =tm\Xk =tn). 

For simplicity, we assume that for any n there are only 
finitely many Pm,n 0- Under these conditions the sequence 
Xo,Xi,... is called a Markov chain with discrete state 
space and of discrete time. Further, we require that Pm,n is 
independent of fc, which means that the chain is homogeneous. 
We always assume discrete time and homogeneity and just say 
discrete-time Markov chain. If the set of states tn, i.e, I ^ Z 
is finite, we call it a finite Markov chain. 

A discrete Markov chain is completely described by its 
initial distribution and its transition probability matrix 

If the distribution of Xk is the row vector Uk = {ak,n)nei, 
then the distribution of Xk+i is a.k+i = CikQ- 

The matrix Q is stochastic, i.e., all entries are non-negative, 
all colunm sums are equal to 1, and no row contains only zeros. 

Markov chains and signal models. We relate a signal 
model as constructed in Sections V, VII, and XII to a discrete 
Markov chain by relating the (time/space) marks to the states 
and the shift matrix to the transition probabiUty matrix Q. 



signal model 


discrete Markov chain 


(time/space) marks t„ 


states tn 


matrix (/)(g) for shift operator q 


probabiUty transition matrix Q 



For example, in the infinite discrete-time model (Figure 1 1), 
(/>(g) is an infinite matrix with I's on the lower diagonal and 
zeros elsewhere, and is thus stochastic, i.e., a candidate Q. The 
states are the time marks tn. Thus, the discrete time model is 
a Markov chain with the following intuitive interpretation: if 
the discrete time is at time t„ (state t„), then, independently 
of the past, it is with probability 1 in the next observation at 
time tn+i- 

Similarly, the matrix (i>{q) corresponding to the space model 
(Figure 14) is stochastic, but this time the transition from a 
location tn in space is with probability 1/2 to or tn+i, 
respectively. This also motivates why we defined the shift 
with the scaling 1/2 (even though omitting the scaUng would 
have led to an equivalent, modified version of Chebyshev 
polynomials), namely, to obtain a stochastic matrix. 

In the generic next neighbor shift (Figure 20), the situation 
is slightly different. To interpret the model as a Markov chain, 
it is necessary, that a„, 6„, c„ > 0. In this case, we can then 



56 



normalize by dividing the nth column of (j){q) by a„ + &„ + c„ 
to make the matrix stochastic. 

We have seen that the realization may modify a model, as in 
the case of the infinite space model given by the C-transform, 
see Figure 16. For the VF-transform, negative numbers occur 
in 4>{q), which destroys the connection to Markov chains in the 
above sense. In the other cases, after possible normalization, 
we obtain again a discrete Markov chain. 

Similar observations hold for finite signal models, which 
correspond to finite Markov chains (finite number of states). 
For example, among the shifts corresponding to the 16 DTTs, 
negative numbers occur for the last row and the last column 
of Table IX, as can be seen from (97) and Table XI. The other 
9 DTTs correspond to signal models that can be interpreted 
as Markov chains. 

Conclusion. The above discussion is somewhat philosophi- 
cal, but the connection is still rigorous in a mathematical sense. 
Note that a discrete Markov chain does not imply linearity, 
which means a non-linear signal model could possibly be built 
from it. 

Also we want to note that the above Markov chain in- 
terpretation is different from a stochastic process or random 
field. In a random field, random variables describe uncertainty 
in the signal and they take values in M or C. In the above 
interpretation, the random variables explain the uncertainty in 
the location (in time or space) and they take values t„. In other 
words, random fields model the signal values s„, whereas 
in the discussion in this section Markov chains model the 
signal indices n. Under certain assumptions to be discussed in 
Section XVI-A, random fields are equivalent to signal models. 



B. Markov Models, Graphs, Matrices, and Polynomial Alge- 
bras 

Above we discussed the relationship between signal models 
and discrete Markov chains. Besides this relation, we repeat- 
edly visualized signal models using graphs (using Definition 2) 
or by the shift matrix 4>{x), i.e., the coordinate matrix for the 
respective shift. Since x generates A, it contains most if not 
all information about the signal model. 

We investigate in this section the exact connection between 
these four concepts. We focus on the case of a finite shift- 
invariant signal model provided by a module C[x]/p{x). The 
four concepts are: 

• a square matrix A S C"X"; 

• a weighted graph with n vertices; 

• a finite Markov chain with n states (see Section XV-A); 

• a shift-invariant signal model {A,A4,^) with A4 — 
C[x]/pix). 

The relationship is displayed in Figure 22 and discussed 
next. Every solid arrow in Figure 22 signifies that one can 
translate the source concept into the target concept. The dashed 
arrows mean that this translation depends on a condition, 
which is then given next to this arrow. As central concept, 
we use the square matrix A. 

Matrix and weighted graph. Every weighted graph 
uniquely determines an adjacency matrix. Conversely, every 



signal model {A, A4, 

A^M^ C[x]/p{x) 

4 

niAix) / 
= Pa{x) ,' 



weighted 
graph 



adjacency matrix 




matrix A 

\ A > 
probability \ 



transition 
matrix 



Markov 
chain 



Fig. 22. The connection between square matrices A, weighted graphs, regular 
signal models, and finite Markov chains. 



matrix A can be viewed as the adjacency matrix of a weighted 
graph. 

Matrix and Markov chain. As discussed in the previous 
Section XV-A, every Markov chain uniquely determines a 
probability transition matrix. Conversely, a given matrix A can 
be viewed as a probability transition matrix only if A > (all 
entries non-negative) and no row is equal to zero. In this case 
the columns can be scaled to have sum 1 to yield a stochastic 
matrix. 

Matrix and regular signal model. Given a regular signal 
model with = A ^ C[x]/p(a;), M. uniquely determines 
the shift matrix A = 0(a;). The converse is the subject of the 
following lemma. 

Lemma 29 Let A E C"^". There exists a polynomial p{x) of 
degree n and a basis of C[x]/p{x) such that ip{x) — A if and 
only if the minimal polynomial iriAix) and the characteristic 
polynomial pa{x) of A coincide: 

mA{x) = PAix). 

In this case, p{x) — pa{x). 

Proof: Let A G C"^" and let mA{x) = pa{x). In this 
case A is similar to the companion matrix C of pa{x) = 
x" + Eo<»<n Aa;' (see [88, p. 187, Theorem 4]), which is 
shown in (49). In other words, A = TCT^^ for a suitable 



invertible matrix T. Defining p — pA and h — (1, x, . 



, ,x" 



as basis in A1 = 'C[x]/p(x), we have 4>(x) ~ C. A base 
change in Ai with T yields the result. 

For the converse, let A £ C"^" and assume that mA{x) ^ 
Pa{x) and that A — (j){x) for some module Ai ~ C[x]/p{x) 
and basis b. Let J be the Jordan normal form of A, J = 
TAT^^. If (f)' is the representation afforded by M with basis 
h' = Tb, then J = (j)'{x). Since vriA 7^ Pa, A has eigenspace 
of at least dimension 2, say, for eigenvalue A. Locating these 
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in J, there are Pk,Pe € b', pk ^ Pe, such that 

xpk = Xpk and xpe = \pt 

^ (x — \)pk = {x — X)pe = mod p 
<^ (x - X)pk = {x- X)pe = p 

^ Pk^Pi 

which is a contradiction. 

Finally, if ^ = C[x]/p{x) and (j>{x) = A, then, since ^ is a 
homomorphism of algebras, = 4>{p{x)) = p(0(a;)) = p{A). 
Thus mA\p and ttia = Pa yields p = p^. ■ 

For example. Lemma 28 shows that for a = 1 and an even 
size n, the matrix (123) has the minimal polynomial r„/2+i — 
Tn/2-i of degree n/2 + 1 < n. Thus, we cannot realize the 
matrix as a (j){x). And indeed, in the alternative space model, 
it is realized as (f>{{x~^ + x)/2), where 4) is the finite time 
model, which is regular. However, this means we can still 
create a signal model that has (123) as a shift matrix, but the 
algebra is smaller than the module, i.e., the module or the 
model is not regular. 

Matrix and non-regular signal model. The previous 
discussion opens a new question: namely, given a matrix A, 
under which condition is it possible to reahze A = (f){q{x)), 
where (j) is the regular representation of a suitably chosen 
regular module M — <C\x\/p{x) with basis h. This relaxed 
requirement (when compared to Lemma 29) allows now for 
non-regular modules, i.e., those with an algebra smaller than 
the module; namely, the algebra is now a subalgebra of 
<C[x\/p{x) generated by a suitable element q{x). 

To answer this question, we first have to understand, under 
which condition a polynomial q{x) G A generates a proper 
subalgebra of ^ = C[x\/p{x). We only consider the case of 
a separable p. 

Lemma 30 Let p{x) be a separable polynomial with zeros a = 
(ao, . . . , a„_i), A = €,[x]/p{x), and let q{x) G A. Then q{x) 
generates the algebra A, if and only if q, viewed as a mapping 
on a, is injective. In other words, if q{ai) ^ q{aj) for i ^ j. 

Proof: The key to the proof is to apply the CRT to A and 
investigate the subalgebra generated by g in the decomposed 
domain. The CRT yields 

A = C[x]/pix)^ C[x]/{x-ak) 

a<k<n 

(now viewed as an isomorphism of algebras). The subalgebra 
{q{x)) generated by q{x) is equal to A if and only if a; e 
{q{x)), i.e., if and only if there is a polynomial r{x) such 
that r{q{x)) = x mod p{x). In the decomposed domain, this 
condition translates into the existence of a polynomial r{x) 
that maps the list ((/(ao), . . . , (?(Q:n-i) onto the list a = 
{ao, . . . , a„_i). This is possible if and only if q is injective 
on a. ■ 
Back to the previous example, from Lemma 28, we have 
A = C[x]/{x"-l),andq{x) = {x-^+x)/2= (a;"-i+a;)/2. 
Since q{x) maps conjugate zeros of a;" — 1 to the same value 
(namely to their real part): 

qicj^) = g(a;r') = cos(2fc7r/n), 



q generates a proper subalgebra. This mapping to the same 
value corresponds precisely to the effect of collapsing fre- 
quency responses discussed in Section XI-B, which can now 
be staled more precisely. Namely, if a regular signal model 
A = M = C[x]/p{x) is given, where p is separable with 
hst of zeros a, and if the algebra of filters is reduced to the 
subalgebra generated by q{x), then the frequency responses 
collapse precisely as determined by the mapping 

(ao, . . . , Q!n-i) 1-^ {q{ao), q{an-i). 

Before we extend Lemma 29, we need another auxiliary 
lemma. 

Lemma 31 Let A e C"^" with minimal polynomial m^(a;). 
Then 

(/>: C[x]/mA{x) C">^", 
X A. 

is a representation of A, i.e., a homomorphism of algebras. 

Proof: Follows from niAiA) =0. ■ 
This lemma provides the algebra A for any matrix A selected 
as shift matrix. The remaining question is under which condi- 
tions there exists a module of the form Ai = C[x]/p{x) that 
affords the representation in Lemma 31? This is answered in 
the next lemma, again only considering a separable p. 

Lemma 32 Let A G c^xn xhere exists a module M = 
C[x]/p{x) with separable p and with regular representation 
cf) (w.r.t. a suitably chosen basis b) such that (j){q{xy) = A, if 
and only if A is diagonalizable. 

In other words, every diagonalizable matrix can become 
a shift matrix, where the shift generates a subalgebra A = 
{q{x)) < C[x]/p{x). The module affording the representation 
isM = C[x]/p{x). 

In this case 

A ^ C[x]/mA{x), 
where niA is the minimal polynomial of A. 

Proof: First assume a regular module M = C[x]/p{x) is 
given with separable p and basis b and afforded representation 
^. Then (j){x) and thus (l){q{x)) is diagonalized by any Fourier 
transform for A4 (Theorem 5). 

For the converse assume A G C"^" is diagonalizable with 
list of eigenvalues /3 = (/?o, ■ • • ,/3n-i)> which may contain 
duplicates. Consider an arbitrary regular module C[x]/p{x) 
with separable p, basis b, and afforded representation (j>. Let 
a = (ao, • • • 5 Q^n-i) be the hst of zeros of p. Since a has no 
duplicates, there is a polynomial q of degree at most n — 1 
with q{ak) = /3fc. Thus (t){q{x)) = q{cf){x)) is similar to A: 
B(f){q(x))B~^ = A with a suitable invertible matrix B. A 
base change in M. yields the result. 

The second assertion follows from Lemma 31. ■ 
The proof is constructive and shows the large degree of free- 
dom in choosing the module: any <C[x\/p{x) with separable 
polynomial p. 
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XVI. Gauss-Markov Random Fields and Signal 
Models 

Signal processing usually adopts either a deterministic point 
of view with linear transforms and filtering as the basic 
processing tools or a stochastic perspective where signals 
are modeled as stochastic processes or random fields. We 
have argued elsewhere [55] that there is an intimate connec- 
tion between these two seemingly disparate approaches. This 
connection establishes that linear transforms like the discrete 
Fourier transform, the discrete cosine transform, or the discrete 
sine transform are the Karhunen-Loeve transforms (KLT) 
associated with a discrete time (or causal) or discrete space 
(or noncausal) random field. In this section we show how 
this connection is articulated by the algebraic theory of signal 
processing: how the signal model in Definition 1 relates to 
Gauss-Markov random fields (GMRF) and how Fourier trans- 
forms, as the generic diagonalizing transforms in the algebraic 
approach, relate to the KLT of suitably defined random fields. 
We start by introducing Gauss-Markov random fields and 
KLTs in Section XVI-A. Then we consider two examples, 
again modeling time and space, that show similarities and 
differences between the algebraic and the stochastic signal 
model. In Section XVI-C, we extend this connection to generic 
results connecting the stochastic and the algebraic approaches. 
We focus on finite, real-valued signal models. 



A. Gauss-Markov Random Fields 

Random fields. Consider the n random variables 
So, - ■ ■ , Sn-i that follow the difference equation 

5fc — ^k^k—m^k—m ~t~ * * * "f- (i'k^k^k~\~ 

\- ak,k+mSk+m + ^k, (134) 

for < fc < n. We call {sfe}o<fe<n a noncausal or acausal 
random field, or random field for short. The a^/ G R are the 
field parameters and the Vk are random variables representing 
the error or noise in the representation. The noise Vk is 
assumed here to be zero mean Gauss, which, because of 
the linearity in (134), makes {sfe}o<fe<n a Gauss random 
field. The noise and signal covariances will be specified 
below. Each signal value Sk depends on at most m right 
and left neighbors up to the uncertainty in vj.. If, for all 
k, afe,fe+i = • • • = ak.k+m = 0, and the noise is white, 
then the random field is causal and usually called a process 
instead of a field. If in (134), the ak,e do not depend on k, 
then the field is called homogeneous, or stationary if it is a 
process. At the boundaries, indices outside the signal index 
scope < fc < n — 1 occur in (134), which poses the 
need for m left and m right boundary conditions that express 
s_i, . . . , S-m and s„, . . . , Sn+m-i as hnear combinations of 
So, • • • , Sn-i- We assume that the boundary conditions (b.c.) 
are already implicit in (134), i.e., the values ak,e have been 
adjusted accordingly. 

After the b.c.'s are fixed, the n equations (134) can be com- 
bined into vector form. Namely, the Sk and Uk are collected 



into random vectors s, u, respectively, to get 

s = As + i', A= [ak,e\o<k,e<n, or 

(I„ -A)s = V. (135) 

This is the initial representation of a random field we will 
work with. 

GMRF: MMSE estimate. We now discuss the covariance 

structure of u, i.e., its covariance matrix E^, to make the field s 
a GMRF and to derive from (134) the minimum mean square 
error (MMSE) representation of the GMRF. We distinguish 
several cases. For simplicity we assume that I„ —A has fuU 
rank, i.e., does not have the eigenvalue 0. 

Case 1: In— A, i.e.. A, is symmetric and In— A is posi- 
tive definite. We assume that the u is correlated noise with 
covariance matrix Y.^, given by 

Y^ = a\l,,-A). 

Then, it is well known, e.g., [80], that (134), or (135), is the 
MMSE representation of an mth-order GMRF and that the 
covariance Ss of the field s, and the field and noise cross 
covariance Ti^^„ are given by 



= a^{ln-A)-\ 

^s,v — O"^ In • 



(136) 



The cross-covariance Ss_^ being diagonal reflects the orthog- 
onality between the field s and the noise v in the MMSE 
representation. 

Case 2: I„ —A, i.e.. A, is symmetric, and I„ —A has negative 
eigenvalues. We now assume that the noise is white with 
covariance 

El^ = (T^ I„ . 

The MMSE representation of the field follows from (135) 
by multiplying both sides by I„ -A"^ = I„ -A. The GMRF 
MMSE representation hence becomes 



{ln-Afs = {ln-A)u. 
The GMRF covariance is now 

T., = a\ln-A)-\ 



(137) 



(138) 



Case 3: I„ —A, i.e.. A, is not symmetric. In this case, we 
assume the noise in (135) to be white with covariance 



Til, = a In • 



(139) 



The MMSE representation follows from (135) by multiplying 
both sides with I„ —A^. As a result we get the signal 
covariance to be 



T, = <j\\n-A)-\ln-A 



T\-l 



(140) 



KLT. The Karhunen-Loeve transform (KLT) is defined in 

statistics as the linear transform that diagonalizes the covari- 
ance of a random vector. For an m-th order GMRF, the KLT 
is a unitary matrix F such that 

FSsF"^ = FH^F-'^ is diagonal, (141) 

where is given by (136) or (138) for a symmetric A, and 
by (140) for a non-symmetric A. 
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The transformed random vector Fs has a diagonal covari- 
ance matrix, i.e., the KLT decomposes, via a base change, a 
signal into its statistically independent components. 

We adopt the terminology in [80] and refer to the inverse 
covariance matrix as the potential matrix; it follows that the 
KLT diagonaUzes the potential matrix. As suggested in [80], 
we will mostly work with the potential matrix. 

B. Examples: Discrete Time and Space 

To understand the similarities and differences between an 
algebraic signal model and a corresponding stochastic model, 
or random field, we consider two examples: discrete time and 
discrete space. The general connection is presented afterwards. 

The correspondence between algebraic and stochastic mod- 
eling is established in an intuitive way by relating the matrix A 
in (135) to the shift, or shift matrix, in the algebraic model. 

Discrete space. We construct a random field that expresses 
the spatial structure given by the space shift (Figure 14) or the 
visualization of the space model in Figure 15. It is given by 
the first-order homogeneous GMRF 

Sk = a{sk-i + Sfe+i) + i^fe, k& Z. (142) 

The noise {vk} has the appropriate covariance structure as 
discussed above. The parameter a could be chosen as 1/2 (as 
in the space shift), but we will see that the exact choice is not 
crucial. 

To construct a finite-length GMRF for the index scope < 
k < n, we need to introduce b.c.'s that express ,s_i and s„ 
as linear combinations of the Sfc, < fc < n. Unfortunately, 
there are no clear guidehnes as to which b.c.'s are natural 
choices. In the literature, these choices are related to the b.c.'s 
used in solving partial differential equations [80], [56]. In the 
algebraic finite space model, we chose the b.c.'s to yield a 
monomial signal extension, but for the GMRF in (142), we 
don't have a notion of signal extension beyond s_i and s„, 
since there is no underlying algebraic structure. 

We now relate (142) to the finite space model to be able to 
draw from the algebraic theory. We choose as an example a 
pair of b.c.'s that is associated to a finite space model, namely 
s_i = So and s„ = s„_i, which leads to 

"11 

1 1 



• 1 

1 1 

Clearly, A, and thus I„ —A, in (135) is symmetric, so we can 
be in either Case 1 or Case 2 above. To find out which case, 
we first determine when 1„ — A is positive definite. To answer 
this question, we apply our algebraic theory. Namely, from 
Table IX and using (97), we know that I„ —A = 0(1 — 2ax), 
where </> is the representation afforded by the regular module 
M = C[x]/{x — l)Un-i with F-basis. The zeros of (x — 
l)Un-i are given by cos(fc7r/n), < A; < n (Table XVI in 



Appendix 111). Using (95), the eigenvalues of I„ —A are hence 
given by the evaluations of 1 — 2ax at these zeros: 

(1 — 2acos(fc7r/n)), < fc < n. 

It follows that the matrix I„ — A is positive definite (Case l)if 
and only if —1/2 < a < 1/2. By (136), the covariance of 
the field s is Eg = (In ~^)^ in this case. The KIT in this 
case is also provided by the algebraic theory and is unique up 
to a unitary diagonal matrix D. It is given by the orthogonal 
DCT-2„. 

If I„ —A has negative eigenvalues (Case 2), i.e., \a\ > 1/2, 
then the covariance of s is provided by (138) as Ss = 
(I„ -A)-^. Thus, the orthogonal DCT-2 is also a KLT in this 
case, but may not be unique (up to a diagonal), since (I„ —A)"^ 
may have duplicate eigenvalues. The cases in which this 
occurs can be computed explicitly by again using (95), which 
provides these eigenvalues as the evaluation of (1 — 2ax)^ at 
cos(fc7r/n), < fc < n. 

We can ask what other GMRFs have the DCT-2 as the KLT. 
To address this question, we consider all matrices diagonalized 
by DCT-2 — these are exactly the matrices A = (t>{h), where 
h G A = C[x]/{x — 1 )[/„_!. In particular, for a fixed m <n, 

let h = 2aiTi -|- 2a2T2 H h 2a„T„. Then A = (f>{h) is 

naturally expressed in terms of (f>{Tk), where Tk are the /c- 
fold space shift operators, and so A is an m-banded matrix 
(only the m upper and lower diagonals are non-zero) besides 
the entries due to the signal extension. The matrix I„ —A is 
positive definite, if and only if 1 — h{x) takes only positive 
values at cos(fc7r/n), < fc < n. Under these conditions, h 
determines a homogeneous mth-order GMRF with monomial 
b.c.'s S-i-i = Si and Sn+i = Sn-i-u < i < m, and the 
KLT is the DCT-2 as desired. 

In this example, up to now, we considered only the matrix A 
in (143). We derived the conditions on a under which the 
DCT-2 is the KLT for a homogeneous symmetric mth-order 
GMRF with monomial b.c.'s s_i_i = .s; and s„+i = 
< i < m. We can adapt this discussion to all 16 DTTs, as 
long as we make sure that the corresponding A is symmetric. 
In our algebraic theory, this leads to the symmetric signal 
models (Definition 18) associated to the orthogonal DTTs (see 
Section VIII-F). As explained in that section, some of the 16 
symmetric models may not be strictly homogeneous, i.e., (142) 
may need to be adjusted for fc = 0, 1 or fc = n — 2, n — 1. 

In summary, we obtain the following theorem, which com- 
pletes the result from [55]. 

Theorem 33 (DTTs as KLTs) Let 1 < m < n. Every DTT„ 
is a KLT for the homogeneous mth-order GMRF 

Sk = ai(sfe-l + Sfe+l) H 1- am(sfc-m + Sk+m) + Vk, 

with monomial b.c.'s given by the signal extension of the 
algebraic signal model associated to the respective DTT 

In the alternative finite space model (Section XI), we iden- 
tified also the periodic boundary conditions s_i = s„_i and 
Sn = So as possible choice for a monomial signal extension. 
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The matrix A then becomes 



1 

1 1 
1 

1 

1 



1 

1 

1 



(144) 



which is also symmetric. If we choose A4 = C[x]/(x" — 1) 
with basis (1, a;, ... , a;"~^) and </> is the representation of ^ = 
M afforded by M, then I„ -A = </)(! - a{x-^ + x)). The 
zeros of x" — 1 are the nth roots of unity, and hence the 
eigenvalues of I„ —A are the evaluations of 1 — a{x~^ + x) 
at these zeros: 

(1 - 2acos(2fc7r/n)), < fc < n/2. 

It follows, similar to above, that 1„ —A is positive definite 
if and only if -1/2 < a < 1/2 for odd n, and -1/2 < 
a < 1/2 for even n. Also, similar to above, we can extend 
this discussion to higher order GMRFs by setting A ~ (t)(l — 

ai{x~^+x^) am{x~"^+x"^)), m < n/2. The algebraic 

theory now also establishes the set of all KLTs for the GMRF 
in this case as the set of matrices 

X . rdft;, 

where RDFT^ is the orthogonal RDFT, and X is any unitary 
x-shaped matrix of the form (70). 

Theorem 34 (DFT and real DFTs as KLTs) Let 1 < m < 
n/2. The orthogonal DFT, RDFT, and DHT are KLTs for the 
homogeneous mth-order GMRF 

Sfe = ai(Sfe_i + Sfe+i) H h am{Sk-m + Sk+m) + ^k, 

with periodic b.c.'s. 

As mentioned before, from a strict computational point of view 
the choice should be the RDFT in this case. 

In summary, the concepts of "algebraic space model" and 
associated "homogeneous symmetric GMRF" are (essentially) 
equivalent. As a consequence, every orthogonal Fourier trans- 
form for the former is a KLT for the latter and vice-versa. 

For the algebraic time model, or causal GMRF, the situation 
is different as we explore next. 

Discrete time. Again, we consider first an example, before 
dealing with the general case in the next subsection. We 
proceed as in the case of discrete space above. The structure 
of the time-shift (Figure 11) is described by the first-order 
causal stationary GMRF (or stochastic process) 



(145) 



The parameter a is left undetermined at this point. The 
noise I'k is white. 

To construct a GMRF for the finite index scope < A; < n, 
we need to define b.c.'s for s_i and s„. We choose the periodic 



b.c. .s_i = Sn identified by the algebraic theory and get as A 
in (135) a scaled cyclic shift matrix: 



(146) 



Since A, and thus I„ — A, is not synmietric, we are in Case 3. 
The covariance of the signal s is given by (140) as Es = 
(I„ —A)~^{ln —A^)~^, which is guaranteed to have no neg- 
ative eigenvalues, regardless of the value of a. Now, however, 
in contradistinction to the finite space model of the previous 
example, there is a difference between the algebraic model and 
the associated GMRF. Namely, between the Fourier transform 
(for the algebraic model), which diagonalizes A, and the KLT 
(for the GMRF), which diagonahzes Sg, or the potential matrix 
(I„ —A'^){In — ^). In general, these diagonalization problems 
are distinct. 

However, with the particular A in (146), we still have a 
close relationship. Since A in (146) is orthogonal, 

S-i = (I„ -A){In -A'^) = (1 + a^) I„ -A', (147) 

where A' is the matrix in (144). Since A' = A+l/a"--'^A"--'^, 
every matrix that diagonalizes A also diagonalizes A', and 
hence Eg. In other words, every unitary Fourier transform 
for the finite algebraic time model (with periodic b.c.'s) is 
a KLT for its GMRF counterpart. The converse does not hold 
in general. For example, the (orthogonal) RDFT is a KLT, but 
not a Fourier transform. 

As an aside, (147) shows that every KLT for the 
GMRF (145) is a KLT for the GMRF (142), if, for both, peri- 
odic b.c.'s are chosen. In other words, the causal GMRF (145) 
and the noncausal GMRF (142) with periodic boundary con- 
ditions are essentially equivalent. 

The above examples for discrete space and discrete time 
convey the essential relationship between the algebraic signal 
model with shift matrix A and the GMRF with matrix A 
in (135). Next, we consider the general case. 

C. GMRFs and Signal Models 

We now consider the relation between the statistical model 
and the algebraic approach in the general case. The connection 
is established by relating the matrix A in (135) to the shift 
matrix in the algebraic model, and is provided by Lemmas 29 
and 32. The diagonalization property of the KLT in (141) 
will be related to the diagonahzation property of the Fourier 
transform in Theorem 5. 

Equivalence in the symmetric case. We identify under 
which conditions GMRFs and signal models are equivalent. 

Theorem 35 The following two concepts of signal models are 
equivalent: A GMRF with symmetric matrix A in (135) and a 
synmietric algebraic signal model {A, M , with shift matrix 
(/)(a;) = A. Further: 

• If 1„ - A is positive definite, the KLT for the GMRF is 
an orthogonal Fourier transform for the signal model, and 
vice-versa. 
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• If I„ —A has negative eigenvalues, then every orthogonal 
Fourier transform is a KLT for the GMRF, but, in general, 
the converse does not hold. Namely, if = R[x]/p{x), 
then the converse holds if and only if the mapping x i— > 
x'^ is injective on the zeros of p. 

Proof: The proof is straightforward. The main point 
is what we noted above that diagonalizing A is equivalent 
(and accomplished by the same matrices) as diagonaUzing 
a\ln-A)-^ in (141). 

Let a GMRF with symmetric matrix A be given, and let 
In— A be positive definite. Then, by (136), a given KLT F 
diagonahzes Sg = (I„ — or, equivalently, A. Since A is 
symmetric it is diagonalizable and we can apply Lemma 32 to 
obtain a signal model with M = R[x]/p{x), A = {q{x)) < 
R[x]/p{x), and (j}{q{x)) = A. Since F diagonalizes A, it has 
to be an orthogonal Fourier transform for this signal model. 

Conversely, let {A,M,<^) with M = M.[x\/p{x) be a real, 
symmetric signal model, and let A < M\x\Jp{x) be generated 
by q{x). Set A = (j){q{x)). Then there is an a > such 
that I„ —aA is positive definite, and a A defines a GMRF via 
(135) with covariance — (Ij^ — o A) ^ . Further, the set of 
all Fourier transforms is the set of all matrices diagonalizing 
A or aA. Since aA is symmetric, it has an orthogonal Fourier 
transform, which is thus a KLT for the GMRF defined by aA. 

To prove the final assertion, we note that, if I„ —aA has 
negative eigenvalues, the covariance of the GMRF s defined 
by A is given by (138) as Ss = (I„ -A)-'^. Thus, the KLT is 
characterized by the diagonalization of (I„ —A)~^. It follows 
that every orthogonal Fourier transform for the algebraic signal 
model (which diagonalizes A) is a KLT, but not vice-versa. 
From Lemma 30 it follows that equivalence holds if and only 
if x'^ is injective on the zeros of p. ■ 

Theorem 35 can be applied to port concepts from the 
algebraic theory to the theory of GMRFs. 

• In Section XVl-B we applied the algebraic theory to 
identify the "good" b.c.'s for an mth-order homogeneous 
GMRF and obtained the 16 DTTs as associated KLTs. 
Further, every DTT was identified as a KLT for GMRFs 
of order 1 < m < n with respect to these b.c.'s. Finally, 
these mth-order GMRF could be conveniently written in 
terms of 

A = <^( ^ hkTk). 

l<k<m 

• We obtain a classification of all possible (variant) sym- 
metric Ist-order GMRFs, which correspond precisely to 
the symmetric GNN models (Section XII-E). In partic- 
ular, this guarantees the existence of fast, 0(nlog^(n)), 
algorithms for all KLTs for these GMRFs (Section XII- 
F). Further, as in the previous item, since the KLT 
diagonahzes the shift matrix A = cp{x), it diagonalizes 
all matrices (f>(q(x)) in the generated algebra, thus a KLT 
for a Ist-order GMRF is automatically also a KLT for all 
higher order GMRFs with suitable b.c.'s. 

• In Section XV, we established the connection between 
signal models, graphs, and Markov chains. This con- 
nection now extends to GMRFs. For example, a GMRF 
with symmetric matrix A in (135) is equivalent to an 



undirected weighted graph. More interestingly, we obtain 
that for A > a GMRF, which models the signal values 
as random variables, is equivalent to a Markov chain, 
which models the signal indices as random variables. 
Connection in the non-symmetric case. Theorem 35 
cannot be extended to the general case of a non-symmetric 
matrix A, since diagonalization of A (as done by the Fourier 
transform for the signal model for which A is a shift matrix) 
is different from the diagonalization of the potential matrix 
(I„ -A)(I„ -A^) (as done by the KLT for the GMRF defined 
by A; w.l.o.g. we omitted fi^). Also, A is not necessary diag- 
onalizable, whereas the symmetric matrix (l^ —A){I„ —A^) 
always is. An example for this situation was the time model in 
Section XVI-B, but in that case, there was still a connection 
between the Fourier transform and the KLT. We establish a 
general condition under which this connection holds. 

First, we remind the reader of the following known property 
[88, p. 272]. 

Lemma 5(5 Let A e R"""". Then is a polynomial in A, 

A^ = q{A), 
if and only if A is normal, i.e., AA^ = A^A. 

Note that normal matrices are precisely those matrices that 
can be diagonalized by a unitary matrix [88, p. 273]. In 
particular, in the real case, (scaled) orthogonal and symmetric 
matrices are normal. 

Now we can extend Theorem 35. 

Theorem 37 Let A e R"^" normal, i.e., = q{A). Further, 
let {A,M.,^) be the signal model for which A is the shift 
matrix, and which is obtained from Lemma 32. Then, this 
model has a unitary Fourier transform, and every such unitary 
Fourier transform is a KLT for the GMRF defined by A via 
(135). The converse holds if and only if the polynomial 

il-x)(l-q{x)) 

generates A, or equivalently (Lemma 30), if (1 — a;)(l — q{x)) 
is an injective mapping on the zeros of p. 

Proof: Let a real, normal matrix A be given, which 
implies A^ ^ q{A) for a suitable polynomial q. Further, let 
{A,M,^), M = M.[x\/p{x), be a signal model for which 
A is the shift matrix, obtained by using Lemma 32. Note 
that this lemma is applicable, since A as a normal matrix 
is diagonahzable. Since A is normal, the signal model has a 
unitary Fourier transform J^. Since diagonahzes A, it also 
diagonalizes (I„ — A)(I„ —q{A)) as a polynomial in A. Thus, 

is a KLT for the GMRF defined by A. 

Conversely, the diagonalization of (I„ — A)(I„ —q{A)) im- 
plies the diagonalization of A only if the polynomial (1 — 
x){l — q{x)) generates A. Using Lemma 30, this is the case 
if this polynomial is injective on the zeros of p. ■ 

Theorem 37 tells us that, for normal A, there may be more 
KLTs for the GMRF than (unitary) Fourier transforms for the 
associated signal model generated by A. This imphes that a 
KLT may be cheaper in terms of arithmetic cost than any 
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of these Fourier transforms. This was precisely the case for 
the time model with periodic b.c.'s in Section XVI-B. The 
algebraic signal model requires as Fourier transform the DFT, 
whereas for the associated GMRF, the RDFT or DHT was 
sufficient. Indeed, the polynomial q in this case is q{x) = 
a;"~^, and the mapping (1 — x){\ — a;"~^) maps conjugate 
roots of unity to the same value and is hence not injective. 

We restricted the discussion of the relationship between 
GMRFs and signal models in our definition (134) to the 
finite case, similar to Section XV. As a consequence, we 
did not discuss the asymptotic equivalence of a GMRF or 
a signal model to its infinite counterpart. However, in the 
case of a Ist-order GMRF corresponding to the GNN model, 
the method in [86] provides a general recipe for asymptotic 
analysis based on orthogonal polynomials, thus, fitting into 
our framework. As a special case, [86] considers the DCT, 
type 2. The analysis shows, in our language, the role of the 
signal extension associated to this transform. In particular, it 
seems the monomiality of the signal extension is crucial for 
convergence. However, with Theorems 33 and 34 in mind, it 
appears that the appropriateness of the chosen b.c.'s should 
have a stronger impact on the transform's performance than 
its asymptotic properties. 

XVII. Conclusions 

We briefly summarize the main contributions of the paper 
and then briefly discuss the further development of the alge- 
braic theory of signal processing that wiU be the subject of 
future publications. 

The algebraic foundation of signal processing. The first 
main goal of the paper is to establish the algebraic structure 
and generalize the fundamental concepts of signal processing. 
Towards this, we determined the following: 

• We introduced the signal model as a triple {A,M,<^) 
of a filter algebra, a signal module, and a bijective linear 
map. We explained how the representation theory of alge- 
bras provides the main ingredients for signal processing 
including the notions of filtering, spectrum, frequency 
response, and Fourier transform. 

• We recognized the fundamental role of the shift as a suit- 
ably chosen generator of the filter algebra and provided a 
general recipe to derive infinite and finite signal models 
from the shift. We used this procedure to derive many 
different signal models for time and space, infinite and 
finite, complex and real, and showed that in each case 
all important signal processing concepts are well-defined 
and available. 

• In particular, by doing so, we generalized the z-transform, 
which is a special case of a signal model, and introduced 
the C-transform as the equivalent of the ^;-transform for 
space models. 

• We identified the equivalence between shift-invariant sys- 
tems and conmiutative algebras. 

Discrete time and space and trigonometric transforms. 
As the second main contribution of this paper and a first 
application of the algebraic theory of signal processing, we 
explained how to derive infinite and finite signal models for 



time and space, and, in doing so, we discovered practically 
all existing trigonometric transforms and completed this class 
with a few new transform not introduced before. We showed 
that each trigonometric transform is the Fourier transform for 
either a time or a space model. The most important results 
include the foUowing: 

• We showed that finite and infinite signal processing are 
instantiations of the same general theory, and explained 
how to derive finite signal models from their infinite 
counterparts. 

• The need for boundary conditions in the finite case proved 
to be central to our theory. In particular, the paper makes 
the role of boundary conditions and their relation to signal 
extensions and signal models completely transparent and 
explains the importance of choosing a simple, that is, 
monomial, signal extension. 

• We showed that time and space signal processing are 
instantiations of the same theory by deriving infinite and 
finite space models in complete analogy, albeit with noted 
important differences, to their time counterparts. As a 
major insight, we showed that the 16 DCTs and DSTs 
are, in a rigorous sense, the space analogue of the DFT. 

• We extended the idea of space models to generic next 
neighbor (GNN) models and showed how their reahzation 
naturally connects to the theory of orthogonal polynomi- 
als. 

Algebraic signal models and GMRFs. Finally, as our third 
main contribution, we established the equivalence (under cer- 
tain conditions) between shift-invariant signal models, graphs, 
and Markov chains, and, perhaps most importantly, Gauss- 
Markov random fields. The connection between these concepts 
makes it possible to look at signal models from different 
perspectives (an important example is the visualization of a 
signal model that we defined) and to understand the inherent 
limitations of the models and thus the signal processing they 
provide. Further, we showed the similarities and differences 
between our signal models and stochastic Gauss-Markov mod- 
els. An important insight was the connection between the 
Fourier transform and the Karhunen-Loeve transform. 

Evolution of the algebraic theory of signal processing. 
This paper shows that signal processing is algebraic in nature. 
In other words, the goal is not to impose a mathematical theory 
on existing signal processing theories, but rather to expose the 
mathematical structure that is the essence or foundation of 
signal processing. In other words, the paper shows that many 
apparently distinct concepts are instantiations of the same 
concepts, for example such as the trigonometric transforms 
as a special examples of Fourier transforms. 

The algebraic theory of signal processing can be used to 
derive new results in signal processing and fast algorithms for 
computing linear transforms. Already in this paper, besides 
the classification of all existing trigonometric transforms, we 
introduced several new transforms. In a future paper [14] we 
use the algebraic theory of signal processing to classify and 
discover fast algorithms. We have started this algebraic theory 
of fast algorithms already in [51], [12]. Besides that, we have 
already used the algebraic theory to derive new signal models 
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and new signal processing schemes. An example is a signal 
model, and its associated Fourier transform, for non-separable 
signal processing on a 2-D triangular grid [3], [15]. We are 
confident that the further development and the application of 
the algebraic theory of signal processing will lead to many 
other new applications. 
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Appendix I 
Algebraic Background 

Algebraic definitions. We provide here the formal defini- 
tions of the most important algebraic concepts used in this 
paper. 

Definition 38 (Algebra) A C-algebra ^ is a ring that is at the 
same time a C-vector space, such that the addition in the ring 
and the addition in the vector space coincide. In addition, for 
a e C and 5, /i e A 



a{gh) = {ag)h = g{ah) 



has to hold. 



Definition 39 (Module) Let >l be a C-algebra. A (left) A- 
module is a C-vector space M. that permits an operation 

Ax M ^ M, {a, m) i-> am, 

which satisfies, for a,b,l G A and m,n G A4, 

a{m + n) = am + an 
{a + b)m = am -j- hm 
{ab)m = a(hm) 
Im = m. 

Definition 40 (Homomorphism of algebras) Let A,B be C- 
algebras. A homomorphism is a mapping cf) : A ^ B that 
satisfies, for a,b G A, a G C, 



(a + 6) 
<t>{ab) 
(j){aa) 



a<j){a). 



Definition 41 (Homomorphism of modules) Let M.,N be A- 
modules. A homomorphism is a mapping ^ : A4 ^ Af that 
satisfies, for a e A m,n G Ai, 

(p{m + n) = (f>{m) + (j){n) 
(j){am) = a4>{m). 
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Vector spaces. We assume the reader is familiar with 
standard linear algebra and only define the notion of direct 
sum that we use in this paper. 

Definition 42 (Direct sum of vector spaces) Let V be a vec- 
tor space, and let U,W < V be subvector spaces with 

V = U + W mdUnW = {0}.ThenV = U + W = U®W 
is called the inner direct sum of U and W. Every element 

V G V can now be uniquely represented as ii = u + w, 
u G U,w G W. We now consider the outer direct sum. 
Represent v as the pair [u, w) and define addition and scalar 
multiplication componentwise. We denote the set of these pairs 
also d& U ®W and call it the outer direct sum of U and 
W . So the outer direct sum of two vector spaces is their 
Cartesian product with componentwise addition and scalar 
multiplication. Every iimer direct sum corresponds to an outer 
direct sum, but the latter is more general, since it can be 
applied to any pair of vector spaces not contained in a common 
larger vector space. 

Chinese remainder tlieorem. The most commonly used 
form of the Chinese remainder theorem (CRT) is for the ring 
of integers. It states that \i n = pq, gcd(p, q) = 1, then 

Z/nZ S Z/pZ X Z/^Z. (148) 

Here, TLjnL denotes the ring of integers {0, . . . , n — 1} with 
addition and multipUcation modulo n, and x denotes the 
Cartesian product with elementwise operation. The isomor- 
phism in (148) is given by, for k G Z/nZ, 

: {k mod p, k mod q). 

In words, the CRT states that "computing (addition and 
multiplication) modulo n is equivalent to computing in parallel 
modulo p and modulo g." 

The CRT also holds for the ring, or algebra, of polynomials 
C[x]. In this case C[x]/p(x)C[x] denotes the ring, or algebra, 
of polynomials of degree less than n with addition and 
multiplication modulo p{x). These polynomial algebras are 
discussed in detail in Section III. We write for short 

C[x\/p{x) = C[x]/p{x)C[x]. 

Theorem 43 ( Chinese remainder theorem for polynomials) 
Let p{x) G C[a;], and let p{x) = r{x)s{x) with 
gcd{r{x),s{x)) = 1. Then 

C[x]/p{x) ^ C[x]/r{x) e C[x]/s{x). 

This isomorphism is given by the mapping 

q{x) 1-^ {q{x) mod r{x),q{x) mod s{x)). 

Note that, for polynomial algebras, we write ® instead of x 
in (148), since they carry the additional structure of a vector 
space, and thus the Cartesian product is equivalent to the outer 
direct sum in Definition 42. 

The Chinese remainder theorem can be generalized to 
the algebra of polynomials in several variables C\x] = 



C[xi, . . . , Xk], but the theory is in general more involved. In 
this case the "factor algebras" have the form 

C[x]/{pi{x)C[x] + ■ ■ ■ + pe{x)C[x]), 

which will write for short as 

C[x]/{pi{x),...,pe{x)). 

Graphs. We introduce some basic notions for graphs that 
we use in this paper. 

Definition 44 (Graph) A weighted, directed graph ^ is a triple 

{V, E, w), where V is the set of vertices, E C V x V the set 
of directed edges, given by ordered pairs, and w : E ^ C a 
weight function that assigns to each edge a complex number. 

Each graph is uniquely described by its square adjacency 
matrix Ag defined as follows. Ag is a complex \V\ x \V\ 
matrix, in which rows and colunans are indexed with V. At 
position {v, w) E V x V, the matrix has the entry u){v, w), if 
{v,w) G E, and else. 



Appendix II 
Module Property of F 

The following theorem shows that £P(Z), 1 < p < oo, is 
an £^ (Z)-module. An analogous statement holds when Z is 
replaced by N. We provide the proof for completeness. 

Theorem 45 Let 1 < p < oo. Then ^p(Z) is an ^i(Z)-module 
with the operation being convolution of sequences. 

Proof: Before we start the proof, we remind the reader of 
Holder's inequality, which for sequences takes the following 
form. Let s G F(Z) and t G £«(Z), where l/p+ l/q = 1. 
Then 

\\s-t\V<\\sU\t\U^ 

where "•" is the pointwise product. We begin the proof of the 
theorem and denote convolution by "★." Let s G £^{Z) and 
h G £^(Z). It is clear that the convolution t = h-k s exists 
and that the result is in e°°{Z), since ^p(Z) C £°°(Z) and h 
is BIBO stable. To prove the Theorem, what we need to show 
is that t = h-k s is also in iP{Z). Using Holder's inequality in 
the third step, we get for the nth output 

feez 

feez feez 
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Raising to the p-th power on each side and summing over Z 
yields 

Ei^-i'' ^ T.(iT.\'r.-kn''k\){Y.\h,\Y-') 
= (Ei^'^ir'- (EE i^-'^i'i'^'^i) 

feez feGZ nez 

= (Ei'^'^i)'-EKi' 

feez nez 

which yields the desired result. In the second step we pulled 
out the last factor in the sum since it does not depend on n. 
In the third step we exchanged the order of summation in the 
second sum (which is absolute convergent). ■ 

Appendix III 
Chebyshev Polynomials 

Chebyshev polynomials, and the more general class of 
orthogonal polynomials, have many interesting properties and 
play an important role in different areas of mathematics, in- 
cluding statistics, approximation theory, and graph theory. An 
excellent introduction to the theory of orthogonal polynomials 
can be found in the books of Chihara, Szego, and Rivlin 
[87], [85], [97]. In this section we give the main properties 
of Chebyshev polynomials that we will use in this paper. 

We call every sequence C = (C„ | n e Z) of polynomials 
that satisfies the three-term recurrence 

C„+i(ar) = 2xCn{x) - C„_i(a;) (149) 

a sequence of Chebyshev polynomials (C stands for Cheby- 
shev). Using (149), the sequence C is uniquely determined 
by the initial polynomials Co,Ci. The most important — and 
commonly known — are the Chebyshev polynomials of theirs/ 
kind, denoted by C„ = T„ and determined by Tq = 1 and 
Ti = X. We provide a few examples: 

r_2 T_i To Ti T2 n 
2x^-1 X 1 X 2x'^ -1 Ax^ - 3x 

For X e [— 1, 1], T„ can be written in closed (or parameterized) 
form as 

Xa = cosn9, cose = x. (150) 
The closed form exhibits the symmetry property, 

T-n=Tr,, (151) 

and can be used to readily derive the zeros of T„, and to show 
that T = {Tn I n € Z) is orthogonal on (—1,1) with respect 
to the weight function w{x) = (1 — a;^)~^/^, i.e., 




Tn{x)Tm{x)w{x)dx = 0, for n ^ m. 



We will occasionally use another parameterization of T, 
which we call power form, given by 



By substituting u = e^^ we obtain (150). 

In this paper, we also consider the Chebyshev polynomials 
of the second, third, and fourth kind, denoted by [/„, Vn,Wn, 
respectively, that arise from different initial polynomials 
Co, Ci. Each of these sequences exhibits a symmetry property 
similar to (151) and is orthogonal on the interval (—1,1) with 
respect to a weight w{x). Furthermore, Un,Vn,Wn can be 
written in closed form. These properties are surmnarized in 
Table XVI. 

In addition, we will need the following properties that are 
shared by all sequences of Chebyshev polynomials including 
T, U, V, W (see [87]). 

Lemma 46 Let C = (C„ | n G Z) be a sequence of 
Chebyshev polynomials. Then the following holds: 

i) The sequence C is determined by any two successive 
polynomials C„, C„+i. 

ii) deg(Co) = 0, deg(Ci) = 1 deg(C„) = n, for n > 0. 

iii) Cn = Ci ■ Un-l — Cq • Un-2- 

iv) Tk ■ Cn = {Cn+k + Cn-k)/2. 

In this paper, we consider only Chebyshev polynomials C 
that satisfy Co = 1, and deg(Ci) = 1. 

The following properties of the four kinds of Chebyshev 
polynomials introduced above are a direct consequence of 
their relationship to cosine and sine function and trigonometric 
identities. The proof is straightforward by induction and is 
omitted. 

Lemma 47 i) The leading coefficient of T„ is 2"~^; the 
leading coefficient of T4, W„ is 2". 

ii) T„(l) = 1, Un{l) =n + l, K(l) = 1, W„(l) = 2n + 1 
iii) T„(-l) = {-l)^,Un{-l) = (-l)"(n + l),y„(-l) = 
(-l)"(2n+l),I^„(-l) = (-l)" 

Appendix IV 
Direct Derivation of Orthogonal DTTs 

In Section VIII-F we derived and explained the orthogonal 
versions of 4 of the 16 DTTs by using the Christoffel-Darboux 
formula (Theorem 15). To do the same for the other 12 DTTs, 
we need to derive variants of (100). We use the DCT-1 as an 
example; the other DTTs are handled similarly. 

The DCT-1 has the associated module C[x]/{Tn - T„_2) 
and base vectors T^. The zeros of r„ — T„_2 = 2(.t^ — 1)?7„-2 
are = cosA:7r/(n — 1), < A: < n. Since the goal is 
to make the top case (x ^ y) in (100) vanish for the zeros 
of Tn — Tn-2, it seems natural to substitute P„ in (100) by 
Tn — Tn-2 to obtain the expression 

A(x, y) = 

Tn-i{y){Tn{x) - r„_2(a:)) - Tn-i{x){Tn{y) - r„_2(j/)) 

x-y 

which vanishes for any choice x = ai,y = Uj, i ^ j. 
Manipulation, including the use of the recurrence (149) for 
Tn, yields 

A{x,y) = 2B{x,y) - 2T„_i(a;)T„_i(j/), with 
, . T„_Uv)T„(x) -T^(v)T„_Ux) 
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TABLE XVI 

Four series of Chebyshev polynomials. The range for the zeros is < A; < n. In the trigonometric closed form cosO = x and in the 

POWER FORM {u + U~'^)/2 = X. 



polynomial n = 0, 1 closed form power form symmetry zeros weight w{x) 





1, 


a; 


cos(n6) 








T^—n — Tn 


COS 


(fc+i)7r 


(l_a;2)-l/2 






2 




n 




Un 


1: 


2x 


Bin(n+l)e 








U-„ = —Un-2 


cos 


(fe + l)7r 


(l-a:2)V2 




sin 6 








n+1 




Vn 


1, 


2a; - 1 


cos(n+^)0 






(n+1/2) 


V-„ =y„_i 


COS 


(fc+i)7r 


(l+a;)V2(i_ 


a;)-i/2 


cos ie 




„1/2_|_„- 


-1/2 






1, 


2a; + 1 


Bin(n+5)e 




»+l/2_„- 


■(n+1/2) 


W-n = -Wn-l 


cos 


(fc + l)T 


(l + a:)-V2(i 


-a;)i/2 


sin ie 




u^/^—u' 


-1/2 


n+i 



or 

B{x,y) = A{x,y)/2 + T„_i{x)Tr,_,{y). (153) 

B{x,y) is the original expression in the top case of 
(100). Equation (153) explains how to adjust (100) to 
yield orthogonality of the transform, namely, by subtracting 
CnTn-i{x)Tn-i{y) on both sides to get 

^ IJLf.^Tk{x)Tk{y) - CnTn-l{x)Tn-l{y) 
0<k<n 

^ ic„/2- A{x,y), Xyty, 
\cn-q{x), x = y, 

with 

q{x) = r„_i(x)r;(x) - r„(a;)r;_i(x) - T„_i(x)r„_i(a;). 

To obtain the actual numbers, we compute /io = tt, fik = 
7r/2, fc > 0. Since occurs in all summands, we can drop 
TT. From Lemma 47, i) (AppendixIII), /?„ = 2"~^, and thus 
c„ = 1, for n > 1. It remains to evaluate q{x) at at = 
coskTr/{n — 1), using = nUn-i- To obtain q{ak) we use 
for fc 7^ 0, n — 1 the closed forms of T„, Un (Table XVI), and 
Lemma 47, ii) else. We get 



q{otk) 



n—l k 0,n — 1, 
2(n-l) A; = 0,n-1. 



Similar to (101), we set 

D = diago<fc<„(g(afc)-^) 

= diag( 2(„_i) , ■ ■■ , 2(n-i) ) 

E = diag(/;ij:^) 

= diag(l,2,...,2,l) 



and VD ■ DCT-1 • VE is orthogonal (Table XII). 



